tooluniverse.utils module

tooluniverse.utils.hf_hub_download(repo_id: str, filename: str, *, subfolder: str | None = None, repo_type: str | None = None, revision: str | None = None, library_name: str | None = None, library_version: str | None = None, cache_dir: str | Path | None = None, local_dir: str | Path | None = None, user_agent: Dict | str | None = None, force_download: bool = False, proxies: Dict | None = None, etag_timeout: float = 10, token: bool | str | None = None, local_files_only: bool = False, headers: Dict[str, str] | None = None, endpoint: str | None = None, resume_download: bool | None = None, force_filename: str | None = None, local_dir_use_symlinks: bool | Literal['auto'] = 'auto') → str[source][source]

Download a given file if it’s not already present in the local cache.

The new cache file layout looks like this: - The cache directory contains one subfolder per repo_id (namespaced by repo type) - inside each repo folder:

  • refs is a list of the latest known revision => commit_hash pairs

  • blobs contains the actual file blobs (identified by their git-sha or sha256, depending on whether they’re LFS files or not)

  • snapshots contains one subfolder per commit, each “commit” contains the subset of the files that have been resolved at that particular commit. Each filename is a symlink to the blob at that particular commit.

[  96]  .
└── [ 160]  models--julien-c--EsperBERTo-small
    ├── [ 160]  blobs
    │   ├── [321M]  403450e234d65943a7dcf7e05a771ce3c92faa84dd07db4ac20f592037a1e4bd
    │   ├── [ 398]  7cb18dc9bafbfcf74629a4b760af1b160957a83e
    │   └── [1.4K]  d7edf6bd2a681fb0175f7735299831ee1b22b812
    ├── [  96]  refs
    │   └── [  40]  main
    └── [ 128]  snapshots
        ├── [ 128]  2439f60ef33a0d46d85da5001d52aeda5b00ce9f
        │   ├── [  52]  README.md -> ../../blobs/d7edf6bd2a681fb0175f7735299831ee1b22b812
        │   └── [  76]  pytorch_model.bin -> ../../blobs/403450e234d65943a7dcf7e05a771ce3c92faa84dd07db4ac20f592037a1e4bd
        └── [ 128]  bbc77c8132af1cc5cf678da3f1ddf2de43606d48
            ├── [  52]  README.md -> ../../blobs/7cb18dc9bafbfcf74629a4b760af1b160957a83e
            └── [  76]  pytorch_model.bin -> ../../blobs/403450e234d65943a7dcf7e05a771ce3c92faa84dd07db4ac20f592037a1e4bd

If local_dir is provided, the file structure from the repo will be replicated in this location. When using this option, the cache_dir will not be used and a .cache/huggingface/ folder will be created at the root of local_dir to store some metadata related to the downloaded files. While this mechanism is not as robust as the main cache-system, it’s optimized for regularly pulling the latest version of a repository.

Parameters:

  • repo_id (str) – A user or an organization name and a repo name separated by a /.

  • filename (str) – The name of the file in the repo.

  • subfolder (str, optional) – An optional value corresponding to a folder inside the model repo.

  • repo_type (str, optional) – Set to "dataset" or "space" if downloading from a dataset or space, None or "model" if downloading from a model. Default is None.

  • revision (str, optional) – An optional Git revision id which can be a branch name, a tag, or a commit hash.

  • library_name (str, optional) – The name of the library to which the object corresponds.

  • library_version (str, optional) – The version of the library.

  • cache_dir (str, Path, optional) – Path to the folder where cached files are stored.

  • local_dir (str or Path, optional) – If provided, the downloaded file will be placed under this directory.

  • user_agent (dict, str, optional) – The user-agent info in the form of a dictionary or a string.

  • force_download (bool, optional, defaults to False) – Whether the file should be downloaded even if it already exists in the local cache.

  • proxies (dict, optional) – Dictionary mapping protocol to the URL of the proxy passed to requests.request.

  • etag_timeout (float, optional, defaults to 10) – When fetching ETag, how many seconds to wait for the server to send data before giving up which is passed to requests.request.

  • token (str, bool, optional) –

    A token to be used for the download.

    • If True, the token is read from the HuggingFace config folder.

    • If a string, it’s used as the authentication token.

  • local_files_only (bool, optional, defaults to False) – If True, avoid downloading the file and return the path to the local cached file if it exists.

  • headers (dict, optional) – Additional headers to be sent with the request.

Returns:

Local path of file or if networking is off, last version of file cached on disk.

Return type:

str

Raises:

  • [RepositoryNotFoundError] – If the repository to download from cannot be found. This may be because it doesn’t exist, or because it is set to private and you do not have access.

  • [RevisionNotFoundError] – If the revision to download from cannot be found.

  • [EntryNotFoundError] – If the file to download cannot be found.

  • [LocalEntryNotFoundError] – If network is disabled or unavailable and file is not found in cache.

  • [EnvironmentError](https – //docs.python.org/3/library/exceptions.html#EnvironmentError) If token=True but the token cannot be found.

  • [OSError](https – //docs.python.org/3/library/exceptions.html#OSError) If ETag cannot be determined.

  • [ValueError](https – //docs.python.org/3/library/exceptions.html#ValueError) If some parameter value is invalid.

class tooluniverse.utils.ModelMetaclass(cls_name: str, bases: tuple[type[Any], ...], namespace: dict[str, Any], __pydantic_generic_metadata__: PydanticGenericMetadata | None = None, __pydantic_reset_parent_namespace__: bool = True, _create_model_module: str | None = None, **kwargs: Any)[source][source]

Bases: ABCMeta

static __new__(mcs, cls_name: str, bases: tuple[type[Any], ...], namespace: dict[str, Any], __pydantic_generic_metadata__: PydanticGenericMetadata | None = None, __pydantic_reset_parent_namespace__: bool = True, _create_model_module: str | None = None, **kwargs: Any) → type[source][source]

Metaclass for creating Pydantic models.

Parameters:

  • cls_name – The name of the class to be created.

  • bases – The base classes of the class to be created.

  • namespace – The attribute dictionary of the class to be created.

  • __pydantic_generic_metadata__ – Metadata for generic models.

  • __pydantic_reset_parent_namespace__ – Reset parent namespace.

  • _create_model_module – The module of the class to be created, if created by create_model.

  • **kwargs – Catch-all for any other keyword arguments.

Returns:

The new class created by the metaclass.

__getattr__(item: str) → Any[source][source]

This is necessary to keep attribute access working for class attribute access.

__instancecheck__(instance: Any) → bool[source][source]

Avoid calling ABC _abc_instancecheck unless we’re pretty sure.

See #3829 and python/cpython#92810

__subclasscheck__(subclass: type[Any]) → bool[source][source]

Avoid calling ABC _abc_subclasscheck unless we’re pretty sure.

See #3829 and python/cpython#92810

tooluniverse.utils.download_from_hf(tool_config)[source][source]
tooluniverse.utils.get_md5(input_str)[source][source]
tooluniverse.utils.yaml_to_dict(yaml_file_path)[source][source]

Convert a YAML file to a dictionary.

Parameters:

yaml_file_path (str) – Path to the YAML file.

Returns:

Dictionary representation of the YAML file content.

Return type:

dict

tooluniverse.utils.read_json_list(file_path)[source][source]

Reads a list of JSON objects from a file.

Parameters: file_path (str): The path to the JSON file.

Returns: list: A list of dictionaries containing the JSON objects.

tooluniverse.utils.evaluate_function_call(tool_definition, function_call)[source][source]
tooluniverse.utils.evaluate_function_call_from_toolbox(toolbox, function_call)[source][source]
tooluniverse.utils.compare_function_calls(pred_function_call, gt_function_call, compare_arguments=True, compare_value=True)[source][source]
tooluniverse.utils.extract_function_call_json(lst, return_message=False, verbose=True, format='llama')[source][source]
tooluniverse.utils.validate_query(query: Dict[str, Any]) → bool[source][source]

Validate a query dictionary for required fields and structure.

Parameters:

query (Dict[str, Any]) – The query dictionary to validate

Returns:

True if query is valid, False otherwise

Return type:

bool

tooluniverse.utils.normalize_gene_symbol(gene_symbol: str) → str[source][source]

Normalize a gene symbol to standard format.

Parameters:

gene_symbol (str) – The gene symbol to normalize

Returns:

Normalized gene symbol

Return type:

str

tooluniverse.utils.format_api_response(response_data: Any, format_type: str = 'json') → str | Dict[str, Any] | List[Any][source][source]

Format API response data into a standardized format.

Parameters:

  • response_data (Any) – The response data to format

  • format_type (str) – The desired output format (‘json’, ‘pretty’, ‘minimal’)

Returns:

Formatted response

Return type:

Union[str, Dict[str, Any]]

tooluniverse.utils.validate_hook_config(config: Dict[str, Any]) → bool[source][source]

Validate hook configuration for correctness and completeness.

This function checks that the hook configuration contains all required fields and that the structure is valid for the hook system.

Parameters:

config (Dict[str, Any]) – Hook configuration to validate

Returns:

True if configuration is valid, False otherwise

Return type:

bool

tooluniverse.utils.validate_hook_conditions(conditions: Dict[str, Any]) → bool[source][source]

Validate hook trigger conditions.

This function checks that the hook conditions are properly structured and contain valid operators and thresholds.

Parameters:

conditions (Dict[str, Any]) – Hook conditions to validate

Returns:

True if conditions are valid, False otherwise

Return type:

bool