tooluniverse.tools.ToolQualityEvaluator module

ToolQualityEvaluator

Evaluates the quality of tool configurations and implementations. Provides detailed scoring and f…

tooluniverse.tools.ToolQualityEvaluator.ToolQualityEvaluator(tool_config, test_cases, evaluation_aspects, *, stream_callback=None, use_cache=False, validate=True)[source]

Evaluates the quality of tool configurations and implementations. Provides detailed scoring and f…

Parameters:

  • tool_config (str) – JSON string of the tool configuration

  • test_cases (str) – JSON string of test cases

  • evaluation_aspects (list[Any]) – Aspects to evaluate (functionality, usability, completeness, best_practices)

  • stream_callback (Callable, optional) – Callback for streaming output

  • use_cache (bool, default False) – Enable caching

  • validate (bool, default True) – Validate parameters

Return type:

dict[str, Any]