CavinHuang-comfyui-nodes-docs/nodes-docs/en-US/SaltAI_Language_Toolkit/docs/LLMGroqModel.md

∞ Groq Model

Documentation

• Class name: LLMGroqModel
• Category: SALT/Language Toolkit/Loaders
• Output node: False

The LLMGroqModel node is designed to interface with the Groq platform for loading and managing language models. It encapsulates the functionality for initializing language models and their corresponding embedding models, providing a streamlined process for integrating Groq's AI capabilities into various applications.

Input types

Required

• model
  • Specifies the name of the language model to be loaded. This parameter is crucial for determining which specific model from the Groq platform will be utilized during the operation.
  • Comfy dtype: COMBO[STRING]
  • Python dtype: str
• groq_api_key
  • The API key required for authenticating with the Groq platform. This key enables secure access to Groq's services and is essential for the operation of the node.
  • Comfy dtype: STRING
  • Python dtype: str
• embedding_model
  • Defines the name of the embedding model to be used in conjunction with the language model. This parameter is important for tasks that require semantic understanding of text, such as similarity searches or text classification.
  • Comfy dtype: COMBO[STRING]
  • Python dtype: str

Optional

• openai_api_key
  • An optional API key for OpenAI services, used when the embedding model is based on OpenAI's technology.
  • Comfy dtype: STRING
  • Python dtype: str

Output types

• llm_model
  • Comfy dtype: LLM_MODEL
  • Returns the initialized language model, including its configuration and state, ready for use in various applications.
  • Python dtype: Dict[str, Any]
• embed_model_only
  • Comfy dtype: LLM_EMBED_MODEL
  • Provides the initialized embedding model separately, including its name, for tasks requiring semantic analysis of text.
  • Python dtype: Dict[str, Any]

Usage tips

• Infra type: CPU
• Common nodes: unknown

Source code

class LLMGroq:
    """
    @Documentation: https://docs.llamaindex.ai/en/stable/api_reference/llms/groq/
    """
    @classmethod
    def INPUT_TYPES(cls):
        return {
            "required": {
                "model": (["llama3-8b-8192", "llama3-70b-8192", "mixtral-8x7b-32768", "gemma-7b-it"],),
                "groq_api_key": ("STRING", {
                    "multiline": False,
                    "dynamicPrompts": False,
                    "default": os.environ.get("GROQ_API_KEY", "")
                }),
                "embedding_model": (
                    sorted([x.value for x in OpenAIEmbeddingModelType]),
                    {"default": "text-embedding-ada-002"},
                ),
            },
            "optional": {
                "openai_api_key": ("STRING", {
                    "multiline": False,
                    "dynamicPrompts": False,
                    "default": os.environ.get("OPENAI_API_KEY", "")
                }),
            }
        }

    RETURN_TYPES = ("LLM_MODEL", "LLM_EMBED_MODEL")
    RETURN_NAMES = ("llm_model", "embed_model_only")

    FUNCTION = "load_model"
    CATEGORY = f"{MENU_NAME}/{SUB_MENU_NAME}/Loaders"

    def load_model(self, model: str, groq_api_key: str, embedding_model:str, openai_api_key:str = None) -> Dict[str, Any]:
        llm = Groq(model=model, api_key=groq_api_key)
        embed_model = OpenAIEmbedding(model_name=embedding_model, api_key=openai_api_key,)
        return ({"llm": llm, "llm_name": model, "embed_model": embed_model, "embed_name": embedding_model}, {"embed_model": embed_model, "embed_name": embedding_model})