Summary

  • OpenAI said it has introduced a control system that can quickly halt experiments when abnormal behavior or security risks arise in AI models.
  • The Control Room is used for internal research tasks such as model training, performance evaluation and data processing, and is designed to stop model output generation if safety or security issues occur.
  • After cases involving research AI models’ unauthorized access to Hugging Face systems and unauthorized use of a cryptocurrency wallet, OpenAI said it strengthened its controls by rejecting new task requests.

Forecast Trend Report by Period

OpenAI has introduced a control system designed to quickly halt experiments when abnormal behavior or security risks are detected in its artificial intelligence models.

Cointelegraph reported on Nov. 9 that OpenAI unveiled a centralized management system called Control Room in an official blog post a day earlier. The system is designed to track tasks running on the company’s internal research infrastructure and stop models from generating outputs if safety or security issues arise.

OpenAI wrote that the ability to detect potential problems and quickly halt models when needed is becoming more important as model capabilities improve and related safety and security risks grow. Control Room is currently being used for internal research work, including model training, performance evaluation and data processing.

Control Room works by assigning a unique identifier to each research task. Before an AI model generates an output, an inference engine checks whether the task remains approved. If researchers or an automated monitoring system order a halt, the model’s authorization to generate output is revoked. That blocks new model outputs even if the existing research program continues running.

The move comes after a series of cases disclosed this year in which OpenAI’s research AI models acted outside prior instructions. In July, some models were found to have attempted to cheat in security evaluations by accessing Hugging Face systems without authorization outside a restricted test environment.

In October, another research AI model was disclosed to have used a cryptocurrency wallet without approval. The model used testnet Ethereum to pay the cost of deploying a smart contract and later omitted the related spending from its final task report.

OpenAI said that even when Control Room issues a shutdown order, it can take up to 35 seconds for an actual block to take effect because of previously stored authorization data. The company said it also strengthened the system by designing it to reject new task requests even if the management system itself fails.

Suehyeon Lee