Nvidia launches new software platform to counter AI issues

Nvidia, the technology giant, has announced the launch of a new platform.

Nvidia is launching a new software platform that will allow AI agents to set safeguards for agents.

The technology giant has announced the launch of the platform shortly after the likes of OpenAI, Anthropic, Meta, and Google saw their AI models attempt to hack other companies.

Justin Boitano, the vice president of enterprise AI at Nvidia, told reporters: “Recent incidents have highlighted a fundamental hurdle for AI agents, and that is that model-level safeguards alone can’t govern what agents can access or do.”

In July, OpenAI’s HuggingFace accessed the open internet and breached Hugging Face, which operates an open-source developer platform.

Reflecting on the controversy, Boitano said: “Each security incident is unique, and we have to look at all of them in detail.

“From what we know, Hugging Face reported over 17,000 agents attacking their infrastructure that went on for days and weeks.”

In July, Anthropic detailed three cybersecurity breaches.

The company reported finding “three incidents in which a Claude model reached the internet from within or while interacting with a third-party evaluation environment, and then gained unauthorised access to the real systems of three different organisations”.

Anthropic also revealed that it launched a review of its cybersecurity in light of recent issues.

The tech firm explained: “On July 21, OpenAI disclosed that several of their models had broken out of an isolated test environment by exploiting a previously unknown (“zero-day”) vulnerability. The models went on to access the production infrastructure of Hugging Face, a platform for open-source machine learning models and AI datasets.

“In response to this incident, we began a large-scale retrospective review of our own cybersecurity evaluations. In particular, we looked for evidence that Claude—like the OpenAI models that accessed Hugging Face—was able to access the internet from within testing environments that should have been sealed off.

“After reviewing 141,006 evaluation runs where Claude could have obtained internet access, we identified three incidents in which a model accessed the internet from within or while interacting with the evaluation environment of Irregular, one of our third-party evaluation partners, and then gained unauthorised access to the production infrastructure of three different organisations.”