The National Institute of Standards and Technology (NIST), the U.S. Department of Commerce agency that develops and tests technologies for the U.S. government, businesses, and the general public, today announced the launch of NIST GenAI, a new program led by the NIST to evaluate generative AI technologies, including text- and image-generating AI.
NIST GenAI is a platform for evaluating various forms of generative AI technology and will publish benchmarks, help develop systems to detect “content authenticity” (i.e. deepfake checking), and promote the development of software to identify the source of fake or misleading information NIST explains on its newly launched NIST GenAI website and in a press release.
“The NIST GenAI program will issue a series of challenge tasks designed to assess and measure the capabilities and limitations of generative AI technologies,” the press release said. “These assessments are used to identify strategies to promote information integrity and guide the safe and responsible use of digital content.”
NIST GenAI’s first project is a pilot study to build systems that can reliably tell the difference between human-generated and AI-generated media, starting with text. (While many services claim to detect deepfakes, studies—and our own testing—have shown that they are unreliable, especially when text is involved.) NIST GenAI invites teams from academia, industry, and research labs to submit either “generators.” – AI content generation systems – or “discriminators” – systems that attempt to identify AI-generated content.
Generators in the study need to generate summaries given a topic and a set of documents, while discriminators need to detect whether a given summary was written by the AI or not. To ensure fairness, NIST will provide GenAI with the data necessary to train generators and discriminators. Systems trained on publicly available data are not accepted, including but not limited to open models such as Metas Llama 3.
Registration for the pilot begins May 1st. The results are scheduled to be published in February 2025.
The launch of NIST GenAI – and the deepfake-focused study – comes at a time when deepfakes are increasing exponentially.
According to Clarity, a deepfake detection company, 900% more deepfakes have been created this year than the same period last year. It understandably sets off alarms. A youngest Opinion poll from YouGov found that 85% of Americans said they were concerned about the spread of misleading deepfakes online.
The launch of NIST GenAI is part of NIST’s response to President Joe Biden’s executive order on AI, which established rules requiring AI companies to be more transparent about how their models work and established a number of new standards, including for Labeling AI-generated content.
It is also NIST’s first AI-related announcement following the appointment of Paul Christiano, a former OpenAI researcher, to the agency’s AI Safety Institute.
Christiano was a controversial choice because of his “Doomerist” views; He once predicted that “there is a 50% chance that AI development could end.” [humanity’s destruction]Critics—including reportedly scientists at NIST—fear that Cristiano could encourage the AI Safety Institute to focus on “fantasy scenarios” rather than realistic, more immediate risks posed by AI.
NIST says NIST GenAI will inform the work of the AI Safety Institute.