tooluniverse.tool_finder_keyword module

Keyword-based Tool Finder - An advanced keyword search tool for finding relevant tools.

This tool provides sophisticated keyword matching functionality using natural language processing techniques including tokenization, stop word removal, stemming, and TF-IDF scoring for improved relevance ranking. It serves as a robust search method when AI-powered search methods are unavailable.

class tooluniverse.tool_finder_keyword.Counter(iterable=None, /, **kwds)

Bases: dict

Dict subclass for counting hashable items. Sometimes called a bag or multiset. Elements are stored as dictionary keys and their counts are stored as dictionary values.

>>> c = Counter('abcdeabcdabcaba')  # count elements from a string

>>> c.most_common(3)                # three most common elements
[('a', 5), ('b', 4), ('c', 3)]
>>> sorted(c)                       # list all unique elements
['a', 'b', 'c', 'd', 'e']
>>> ''.join(sorted(c.elements()))   # list elements with repetitions
'aaaaabbbbcccdde'
>>> sum(c.values())                 # total of all counts
15

>>> c['a']                          # count of letter 'a'
5
>>> for elem in 'shazam':           # update counts from an iterable
...     c[elem] += 1                # by adding 1 to each element's count
>>> c['a']                          # now there are seven 'a'
7
>>> del c['b']                      # remove all 'b'
>>> c['b']                          # now there are zero 'b'
0

>>> d = Counter('simsalabim')       # make another counter
>>> c.update(d)                     # add in the second counter
>>> c['a']                          # now there are nine 'a'
9

>>> c.clear()                       # empty the counter
>>> c
Counter()

Note: If a count is set to zero or reduced to zero, it will remain in the counter until the entry is deleted or the counter is cleared:

>>> c = Counter('aaabbc')
>>> c['b'] -= 2                     # reduce the count of 'b' by two
>>> c.most_common()                 # 'b' is still in, but its count is zero
[('a', 3), ('c', 1), ('b', 0)]

__init__(iterable=None, /, **kwds)

Create a new, empty Counter object. And if given, count elements from an input iterable. Or, initialize the count from another mapping of elements to their counts.

>>> c = Counter()                           # a new, empty counter
>>> c = Counter('gallahad')                 # a new counter from an iterable
>>> c = Counter({'a': 4, 'b': 2})           # a new counter from a mapping
>>> c = Counter(a=4, b=2)                   # a new counter from keyword args

__missing__(key)

The count of elements not in the Counter is zero.

total()

Sum of the counts

most_common(n=None)

List the n most common elements and their counts from the most common to the least. If n is None, then list all element counts.

>>> Counter('abracadabra').most_common(3)
[('a', 5), ('b', 2), ('r', 2)]

elements()

Iterator over elements repeating each as many times as its count.

>>> c = Counter('ABCABC')
>>> sorted(c.elements())
['A', 'A', 'B', 'B', 'C', 'C']

# Knuth’s example for prime factors of 1836: 2**2 * 3**3 * 17**1 >>> prime_factors = Counter({2: 2, 3: 3, 17: 1}) >>> product = 1 >>> for factor in prime_factors.elements(): # loop over factors … product *= factor # and multiply them >>> product 1836

Note, if an element’s count has been set to zero or is a negative number, elements() will ignore it.

classmethod fromkeys(iterable, v=None)

Create a new dictionary with keys from iterable and values set to value.

update(iterable=None, /, **kwds)

Like dict.update() but add counts instead of replacing them.

Source can be an iterable, a dictionary, or another Counter instance.

>>> c = Counter('which')
>>> c.update('witch')           # add elements from another iterable
>>> d = Counter('watch')
>>> c.update(d)                 # add elements from another counter
>>> c['h']                      # four 'h' in which, witch, and watch
4

subtract(iterable=None, /, **kwds)

Like dict.update() but subtracts counts instead of replacing them. Counts can be reduced below zero. Both the inputs and outputs are allowed to contain zero and negative counts.

Source can be an iterable, a dictionary, or another Counter instance.

>>> c = Counter('which')
>>> c.subtract('witch')             # subtract elements from another iterable
>>> c.subtract(Counter('watch'))    # subtract elements from another counter
>>> c['h']                          # 2 in which, minus 1 in witch, minus 1 in watch
0
>>> c['w']                          # 1 in which, minus 1 in witch, minus 1 in watch
-1

copy()

Return a shallow copy.

__delitem__(elem)

Like dict.__delitem__() but does not raise KeyError for missing values.

__eq__(other)

True if all counts agree. Missing counts are treated as zero.

__ne__(other)

True if any counts disagree. Missing counts are treated as zero.

__le__(other)

True if all counts in self are a subset of those in other.

__lt__(other)

True if all counts in self are a proper subset of those in other.

__ge__(other)

True if all counts in self are a superset of those in other.

__gt__(other)

True if all counts in self are a proper superset of those in other.

__repr__()

Return repr(self).

__add__(other)

Add counts from two counters.

>>> Counter('abbb') + Counter('bcc')
Counter({'b': 4, 'c': 2, 'a': 1})

__sub__(other)

Subtract count, but keep only results with positive counts.

>>> Counter('abbbc') - Counter('bccd')
Counter({'b': 2, 'a': 1})

__or__(other)

Union is the maximum of value in either of the input counters.

>>> Counter('abbb') | Counter('bcc')
Counter({'b': 3, 'c': 2, 'a': 1})

__and__(other)

Intersection is the minimum of corresponding counts.

>>> Counter('abbb') & Counter('bcc')
Counter({'b': 1})

__pos__()

Adds an empty counter, effectively stripping negative and zero counts

__neg__()

Subtracts from an empty counter. Strips positive and zero counts, and flips the sign on negative counts.

__iadd__(other)

Inplace add from another counter, keeping only positive counts.

>>> c = Counter('abbb')
>>> c += Counter('bcc')
>>> c
Counter({'b': 4, 'c': 2, 'a': 1})

__isub__(other)

Inplace subtract counter, but keep only results with positive counts.

>>> c = Counter('abbbc')
>>> c -= Counter('bccd')
>>> c
Counter({'b': 2, 'a': 1})

__ior__(other)

Inplace union is the maximum of value from either counter.

>>> c = Counter('abbb')
>>> c |= Counter('bcc')
>>> c
Counter({'b': 3, 'c': 2, 'a': 1})

__iand__(other)

Inplace intersection is the minimum of corresponding counts.

>>> c = Counter('abbb')
>>> c &= Counter('bcc')
>>> c
Counter({'b': 1})

class tooluniverse.tool_finder_keyword.defaultdict

Bases: dict

defaultdict(default_factory=None, /, […]) –> dict with default factory

The default factory is called without arguments to produce a new value when a key is not present, in __getitem__ only. A defaultdict compares equal to a dict with the same items. All remaining arguments are treated the same as if they were passed to the dict constructor, including keyword arguments.

__init__(*args, **kwargs)

__repr__()

Return repr(self).

copy() → a shallow copy of D.

default_factory

Factory for default value called by __missing__().

class tooluniverse.tool_finder_keyword.BaseTool(tool_config)

Bases: object

__init__(tool_config)

classmethod get_default_config_file()

Get the path to the default configuration file for this tool type.

This method uses a robust path resolution strategy that works across different installation scenarios:

1. Installed packages: Uses importlib.resources for proper package resource access

2. Development mode: Falls back to file-based path resolution

3. Legacy Python: Handles importlib.resources and importlib_resources

Override this method in subclasses to specify a custom defaults file.

Returns:

Path or resource object pointing to the defaults file

classmethod load_defaults_from_file()

Load defaults from the configuration file

run(arguments=None)

Execute the tool.

The default BaseTool implementation accepts an optional arguments mapping to align with most concrete tool implementations which expect a dictionary of inputs.

check_function_call(function_call_json)

get_required_parameters()

Retrieve required parameters from the endpoint definition. Returns: list: List of required parameters for the given endpoint.

tooluniverse.tool_finder_keyword.register_tool(tool_type_name=None, config=None)

Decorator to automatically register tool classes and their configs.

Usage:

@register_tool(‘CustomToolName’, config={…}) class MyTool:

pass

class tooluniverse.tool_finder_keyword.ToolFinderKeyword(tool_config, tooluniverse=None)

Bases: BaseTool

Advanced keyword-based tool finder that uses sophisticated text processing and TF-IDF scoring.

This class implements natural language processing techniques for tool discovery including: - Tokenization and normalization - Stop word removal - Basic stemming - TF-IDF relevance scoring - Semantic phrase matching

The search operates by parsing user queries to extract key terms, processing them through NLP pipelines, and matching against pre-built indices of tool metadata for efficient and relevant tool discovery.

STOP_WORDS = {'a', 'all', 'an', 'and', 'any', 'are', 'as', 'at', 'be', 'boy', 'but', 'by', 'came', 'can', 'day', 'did', 'do', 'down', 'each', 'find', 'for', 'from', 'get', 'had', 'has', 'have', 'he', 'how', 'if', 'in', 'is', 'it', 'its', 'long', 'made', 'may', 'new', 'no', 'now', 'number', 'of', 'old', 'on', 'part', 'said', 'see', 'that', 'the', 'their', 'they', 'this', 'time', 'to', 'two', 'up', 'use', 'was', 'way', 'what', 'which', 'who', 'will', 'with', 'your'}

STEMMING_RULES = [('ies', 'y'), ('ied', 'y'), ('ying', 'y'), ('ing', ''), ('ly', ''), ('ed', ''), ('ies', 'y'), ('ier', 'y'), ('iest', 'y'), ('s', ''), ('es', ''), ('er', ''), ('est', ''), ('tion', 't'), ('sion', 's'), ('ness', ''), ('ment', ''), ('able', ''), ('ible', ''), ('ful', ''), ('less', ''), ('ous', ''), ('ive', ''), ('al', ''), ('ic', ''), ('ize', ''), ('ise', ''), ('ate', ''), ('fy', ''), ('ify', '')]

__init__(tool_config, tooluniverse=None)

Initialize the Advanced Keyword-based Tool Finder.

Parameters:

• tool_config (dict) – Configuration dictionary for the tool

• tooluniverse – Reference to the ToolUniverse instance containing all tools

find_tools(message=None, picked_tool_names=None, rag_num=5, return_call_result=False, categories=None)

Find relevant tools based on a message or pre-selected tool names.

This method matches the interface of other tool finders to ensure seamless replacement. It uses keyword-based search instead of embedding similarity.

Parameters:

• message (str, optional) – Query message to find tools for. Required if picked_tool_names is None.

• picked_tool_names (list, optional) – Pre-selected tool names to process. Required if message is None.

• rag_num (int, optional) – Number of tools to return after filtering. Defaults to 5.

• return_call_result (bool, optional) – If True, returns both prompts and tool names. Defaults to False.

• categories (list, optional) – List of tool categories to filter by.

Returns:

• If return_call_result is False: Tool prompts as a formatted string

• If return_call_result is True: Tuple of (tool_prompts, tool_names)

Return type:

str or tuple

Raises:

AssertionError – If both message and picked_tool_names are None

run(arguments)

Find tools using advanced keyword-based search with NLP processing and TF-IDF scoring.

This method provides a unified interface compatible with other tool finders.

Parameters:

arguments (dict) – Dictionary containing: - description (str): Search query string (unified parameter name) - categories (list, optional): List of categories to filter by - limit (int, optional): Maximum number of results to return (default: 10) - picked_tool_names (list, optional): Pre-selected tool names to process - return_call_result (bool, optional): Whether to return both prompts and names. Defaults to False.

Returns:

• If return_call_result is False: Tool prompts as a formatted string

• If return_call_result is True: Tuple of (tool_prompts, tool_names)

Return type:

str or tuple