datadings.writer package

class datadings.writer.FileWriter(outfile, buffering=0, overwrite=False, **kwargs)

Bases: datadings.writer.Writer

Writer for file-based datasets. Requires sample dicts with a unique "key" value.

write(sample)

Write a sample to the dataset file.

Parameters

args – Sample data to write.

class datadings.writer.RawWriter(outfile, buffering=0, overwrite=False, **kwargs)

Bases: datadings.writer.Writer

Writer for raw data. No packing is done. write() requires key and data as arguments.

write(key, data)

Write a sample to the dataset file.

Parameters

args – Sample data to write.

class datadings.writer.Writer(outfile, buffering=0, overwrite=False, **kwargs)

Bases: object

Writers can be used to create dataset files along with index and MD5 hash.

Writer is an abstract class. It cannot be instantiated. Subclasses must implement the abstract write method.

It is recommended to use writers as context manager in “with” statements:

with Writer(‘dataset.msgpack’) as writer:

for sample in samples:

writer.write(sample)

The writer is then automatically closed and index and md5 files are written.

Important

If overwrite is False, the user will be prompted to overwrite an existing file. The user can now:

• Accept to overwrite the file.

• Decline, which raises a FileExistsError. The program should continue as if writing had finished.

• Abort, which raises a KeyboardInterrupt. The program should abort immediately.

Parameters

• outfile – Path to the dataset file.

• overwrite – If outfile exists, force overwriting.

• kwargs – Keyword arguments for datadings.tools.make_printer().

close()

Flush and close the dataset file and write index and MD5 files.

abstract write(*args)

Write a sample to the dataset file.

Parameters

args – Sample data to write.