datadings.commands package

A number of useful tools are installed with datadings. These will be accessible on the command line as datadings-* where * is replaced with one of the submodule names.

The main tool to interact with datadings in this way is datadings-write. It finds available datasets and runs their writing scripts.

These are the available tools:

• datadings-write creates new dataset files.

• datadings-cat prints the (abbreviated) contents of a dataset file.

• datadings-shuffle shuffles an existing dataset file.

• datadings-merge merges two or more dataset files.

• datadings-split splits a dataset file into two or more subsets.

• datadings-bench runs some basic read performance benchmarks.

You can either call them directly or run them as modules with python -m datadings.commands.*, again with star replaced by the name the command, e.g., write.

Submodules

• datadings.commands.bench module
  • Positional arguments
  • bench()
  • entry()
  • main()
  • reader_directory()
  • reader_msgpack()
• datadings.commands.cat module
  • Positional arguments
  • cat()
  • entry()
  • main()
• datadings.commands.convert_index module
  • Positional arguments
  • convert_index()
  • entry()
  • main()
• datadings.commands.merge module
  • Positional arguments
  • cumsum()
  • entry()
  • main()
  • merge_concat()
  • merge_random()
  • select_set()
  • setup_ranges()
• datadings.commands.sample module
  • Positional arguments
  • entry()
  • main()
  • sample()
• datadings.commands.shuffle module
  • Positional arguments
  • entry()
  • main()
  • shuffle()
• datadings.commands.split module
  • Positional arguments
  • entry()
  • main()
  • split_dataset()
• datadings.commands.write module
  • Positional arguments
  • entry()
  • find_writers()
  • format_writers()
  • main()
  • writer_link()