tlpipe.container.container.BasicTod

class tlpipe.container.container.BasicTod(files=None, mode='r', start=0, stop=None, dist_axis=0, use_hints=True, comm=None)

Basic time ordered data container.

Inherits from caput.memh5.MemDiskGroup.

Basic one-level time ordered data container that allows any number of datasets in the root group but no nesting.

This container is intended to be an base class for other concrete data classes, so only basic input/output and a limited operations are provided. Usually you should not use this class directly, use a concrete sub-class instead.

Parameters:

• files (None, string or list of strings) – File name or a list of file names that data will be loaded from. No files if None. Default None.
• mode (string, optional) – In which mode to open the input files. Default ‘r’ to open files read only.
• start (integer, optional) – Starting time point to load. Non-negative integer is relative to the first time point of the first file, negative integer is relative to the last time point of the last file. Default 0 is from the start of the first file.
• stop (None or integer, optional) – Stopping time point to load. Non-negative integer is relative to the first time point of the first file, negative integer is relative to the last time point of the last file. Default None is to the end of the last file.
• dist_axis (string or integer, optional) – Axis along which the main data is distributed.
• use_hints (bool, optional) – If True, will try to use the hints in the first file of files to construct this class. Default True.
• comm (None or MPI.Comm, optional) – MPI Communicator to distributed over. Default None to use mpiutil._comm.

Attributes

attrs	Attributes attached to this object.
comm	Reference to the MPI communicator.
dist_axis	Convenience for self.main_data_dist_axis.
dist_axis_name	Name of self.main_data_dist_axis.
distributed
file	Not a file at all but the top most MemGroup of the tree.
history	The analysis history for this data.
main_axes_ordered_datasets	Datasets that have axis aligned with the main data.
main_data	The main data in the container which is a convenience for self[self.main_data_name].
main_data_axes	Axes of the main data.
main_data_name	Main data in the data container.
main_time_ordered_datasets	Datasets that have the first axis aligned with the main data.
name	String giving the full path to this entry.
ondisk	Whether the data is stored on disk as opposed to in memory.
parent	Parent MemGroup that contains this group.
time_ordered_attrs	Attributes that are different in different files.
time_ordered_datasets	Time ordered datasets.

__init__(files=None, mode='r', start=0, stop=None, dist_axis=0, use_hints=True, comm=None)

Methods

add_history([history])	Create a new history entry.
all_data_operate(func[, copy_data])	Operation to the whole main data.
check_status()	Check that data hold in this container is consistent.
close()	Close underlying hdf5 file if on disk.
copy()	Return a deep copy of this container.
create_dataset(name, \*args, \*\*kwargs)	Create and return a new dataset.
create_group(name)	Create and return a new group.
create_main_axis_ordered_dataset(axis, name, ...)	Create a axis_name ordered dataset.
create_main_data(data[, recreate, copy_attrs])	Create or recreate a main dataset.
create_main_time_ordered_dataset(name, data)	Create a main type time ordered dataset.
create_time_ordered_dataset(name, data[, ...])	Create a time ordered dataset.
data_operate(func[, op_axis, axis_vals, ...])	A basic data operation interface.
data_select(axis, value)	Select data to be loaded from input files along the specified axis.
dataset_common_to_distributed(name[, ...])	Convert a common dataset to a distributed one.
dataset_distributed_to_common(name)	Convert a distributed dataset to a common one.
dataset_name_allowed(name)	Datasets may only be created and accessed in the root level group.
delete_a_dataset(name)	Delete a dataset and also remove it from the hint if it is in it.
delete_an_attribute(name)	Delete an attribute and also remove it from the hint if it is in it.
flush()	Flush the buffers of the underlying hdf5 file if on disk.
from_file(file_[, ondisk, distributed, ...])	Create data object from analysis hdf5 file, store in memory or on disk.
from_group([data_group, detect_subclass])	Create data object from a given group.
get((k[,d]) -> D[k] if k in D, ...)
group_name_allowed(name)	No groups are exposed to the user.
info()	List basic information of the data hold by this container.
items(() -> list of D’s (key, value) pairs, ...)
iteritems(() -> an iterator over the (key, ...)
iterkeys(() -> an iterator over the keys of D)
itervalues(...)
keys(() -> list of D’s keys)
load_all()	Load all attributes and datasets from files.
load_common_attrs()	Load common attributes from the first file.
load_common_datasets()	Load common datasets from the first file.
load_main_axes_excl_main_data()	Load main axes ordered datasets (exclude the main data).
load_main_axes_ordered_data()	Load main axes ordered dataset.
load_main_data()	Load main data from all files.
load_time_ordered_attrs()	Load time ordered attributes from all files.
load_time_ordered_datasets()	Load time ordered datasets (excepts those also in main_axes_ordered_datasets) from all files.
redistribute(dist_axis)	Redistribute the main time ordered dataset along a specified axis.
require_dataset(name, shape, dtype, \*\*kwargs)	Require a dataset to exist, create if it doesn’t.
require_group(name)	Require a group to exist, create if it doesn’t.
save(filename, \*\*kwargs)	Save data to hdf5 file.
subset()	Return a subset of the data as a new data container.
subset_select(axis, value)	Select a subset of the data along the specified axis.
to_disk(filename, \*\*kwargs)	Return a version of this data that lives on disk.
to_files(outfiles[, exclude, check_status, ...])	Save the data hold in this container to files.
to_memory()	Return a version of this data that lives in memory.
values(() -> list of D’s values)