niimpy.reading.read module

Read data from various formats, user entery point.

This module contains various functions read_* which load data from different formats into pandas.DataFrame:s. As a side effect, it provides the authoritative information on how incoming data is converted to dataframes.

niimpy.reading.read.read_csv(filename, read_csv_options={}, add_group=None, tz=None)[source]

Read DataFrame from csv file

This will read data from a csv file and then process the result with niimpy.util.df_normalize.

Parameters:
filenamestr

filename of csv file

read_csv_options: dict

Dictionary of options to pandas.read_csv, if this is necessary for custom csv files.

add_groupobject

If given, add a ‘group’ column with all values set to this.

niimpy.reading.read.read_csv_string(string, tz=None)[source]

Parse a string containing CSV and return dataframe

This should not be used for serious reading of CSV from disk, but can be useful for tests and examples. Various CSV reading options are turned on in order to be better for examples:

  • Allow comments in the CSV file

  • Remove the datetime column (redundant with index but some older functions break without it, so default readers need to leave it).

Parameters:
stringstring containing CSV file
Returns:
df: pandas.DataFrame
niimpy.reading.read.read_sqlite(filename, table, add_group=None, user=<class 'niimpy.reading.database.ALL'>, limit=None, offset=None, start=None, end=None, tz=None)[source]

Read DataFrame from sqlite3 database

This will read data from a sqlite3 file, taking sensor data in a given table, and optionally apply various limits.

Parameters:
filenamestr

filename of sqlite3 database

tablestr

table name of data within the database

add_groupobject

If given, add a ‘group’ column with all values set to this.

userstr or database.ALL, optional

If given, return only data matching this user (based an column ‘user’)

limitint, optional

If given, return only this many rows

offsetint, optional

When used with limit, skip this many lines at the beginning

startint or float or str or datetime.datetime, optional

If given, limit to this starting time. Formats can be int/float (unixtime), string (parsed with dateutil.parser.parser, or datetime.datetime.

endint or float or str or datetime.datetime, optional

Same meaning as ‘start’, but for end time

niimpy.reading.read.read_sqlite_tables(filename)[source]

Return names of all tables in this database

Return a set of all tables contained in this database. This may be useful when you need to see what data is available within a database.