Innodata Launches Open-Source Toolkit for LLM Safety Evaluation

Innodata Inc. (NASDAQ:INOD), a pioneer in data engineering, announced the release of an open-source LLM Evaluation Toolkit and 14 evaluation datasets on April 25, 2024. Designed for enterprise use, these tools aim to enhance the safety evaluation of Large Language Models (LLMs) by enabling simultaneous testing across various harm categories.

The toolkit facilitates the identification of specific input conditions that produce harmful outputs. This will allow developers to better understand and adjust their AI systems to meet intended safety standards. Innodata has made these resources available on GitHub, encouraging immediate use by enterprises and plans to offer a commercial version later this year.

In conjunction, Innodata has also released a research paper detailing their methods for benchmarking LLMs against benchmarks such as factuality, toxicity, bias, and hallucination tendencies. Significant use cases include evaluations on models like Llama2 and GPT. This initiative is part of Innodata’s broader efforts since Q4-2023 to work on trust and safety for large tech firms, with multiple new engagements in Q1-2024 emphasizing the industry's emphasis on responsible AI development.