src.test

Module contents

class src.test.EvaluationRunner(testers, algorithm_config, dataset, output_path, max_workers=1, use_blocking_executor=False)

Bases: object

Class for efficiently evaluating an algorithm across multiple threads for every mixture in a dataset. If the algorithm requires something that must happen in a single thread, then the execution of the algorithm can happen in two steps - once on a blocking thread, and then continue on a parallel thread by specifying use_blocking_executor as True:

# run on a blocking executor (e.g. on one GPU), in self.blocking_func
features = algorithm.extract_features()
# run the rest on the main executor, using the result self.blocking_func
# this is in self.run_func
algorithm.run(features=features)
algorithm.make_audio_signals()

This class is built assuming that you are using NUSSL style algorithms for source separation. If you’re evaluating on other tasks (e.g. sound classification or music transcription), then you can use new testers or edit this class to your liking. If you’re editing the class, your new code likely belongs in either main_func, run_func, or blocking_func.

Parameters

• testers ([type]) – [description]

• algorithm_config ([type]) – [description]

• dataset ([type]) – [description]

• output_path ([type]) – [description]

• max_workers (int, optional) – [description]. Defaults to 1.

• use_blocking_executor (bool, optional) – [description]. Defaults to False.

blocking_func(file_path)

[summary]

[extended_summary]

Parameters

file_path ([type]) – [description]

Returns

[description]

Return type

[type]

log_scores(scores)

[summary]

[extended_summary]

Parameters

scores ([type]) – [description]

main_func(file_path)

[summary]

[extended_summary]

Parameters

file_path ([type]) – [description]

run()

[summary]

[extended_summary]

run_func(file_path, data=None)

[summary]

[extended_summary]

Parameters

• file_path ([type]) – [description]

• data ([type], optional) – [description]. Defaults to None.