About
AI safety standards around the world need to keep up with the fast pace of development and deployment of AI technology. Our mission is to help accelerate AI safety standards writing.
Formal AI safety standards can play an important role in supporting government initiatives for AI technology regulation. We therefore aim to accelerate such formal standards writing in organizations like ISO/IEC, CEN-CENELEC, the IEEE, and NIST.
We are structured as a virtual lab, bringing together volunteers from all over the world, who self-organize to pursue the above mission.
Current project
Our current project is shown in the diagram below. We convert insights from the existing literature into ready-made pieces of text, which formal standards efforts can include into their AI safety standards documents.
We are positioning ourselves as politically and geographically neutral. As a group, we are not directly engaging with different government initiatives to regulate local or global AI markets.
We aim to support at least the following three government-related standards initiatives:
AI safety standards writing in CEN-CENELEC JTC21, in support of the upcoming EU AI Act
AI safety standards writing in the US NIST AI risk management framework, such as the announced NIST Public Working Group on AI and the Berkeley CLTC initiative for a Profile for Increasingly Multi- or General-Purpose AI
Any AI safety standards effort which might come out of the UK foundation model taskforce and the UK AI safety summit
The data flow in the above diagram, from our lab into formal standards writing efforts, is enabled because we have several members who also active in these formal standards writing efforts. Using the terminology of the standards world, the outputs of this lab can be packaged and submitted as technical expert contributions to the respective standards efforts. It will then be up to these standards efforts themselves to further decide if and how to incorporate these contributions into their eventual standards.
Upcoming positions: Research Analyst
We are expanding and are looking forward to adding 2-3 Research Analysts to our team. You can find more information on the position here.
Applications are closed - applications were due before January 21st, 2024.
Open call for volunteers
We are now open to new volunteers to work with us in the lab. Volunteers can start with a low time commitment (2-5h/week), by committing to review and comment on existing standards contributions we are developing.
A next step, requiring a larger time commitment (5-10h/week), is to author or co-author a new contribution, by leveraging some part of the safety literature and using our style guide. Authors will need to have some basic knowledge of AI technology and safety or quality engineering, but they do not need to be subject matter experts in the literature concerned. We of course also welcome contributions from volunteers who are deep subject matter experts in a specific field, and who want to convert the latest insights from their field into standards text.
Also, we welcome volunteers who want to use our lab as a potential stepping stone towards joining a formal standards organization to participate in a formal AI safety standards writing effort.
All our work happens in a closed workspace, and all discussions happen under the Chatham House Rule.
Differences with formal standards writing efforts
Formal standards writing efforts tend to have very high barriers to entry. They often require that organizational fees are paid before a volunteer can participate. To qualify for a government mandate, some efforts must also apply geographical restrictions to who can join.
Furthermore, for a volunteer to contribute effectively to a formal standards effort, a large time investment is often needed to understand all standards process details. It can also take a lot of effort to locate the right subcommittee to make specific contributions to, and the right moment in time to make them.
We have set up the AI Standards Lab to have much lower barriers to entry for volunteers and subject matter experts around the world. This allows us to accelerate standards writing by accessing a pool of volunteer labor that is simply not available to the formal standards efforts.
Another important difference with formal standards writing is that in this lab, we focus only on the topic of encoding the state of the art in AI risk management. Formal standards writing efforts have a much broader scope, they need to resolve a whole range of other difficult legal-technical and regulatory questions too, in consultation with their respective government(s).
The formal standards efforts we target also usually work under Chatham house rules, supplemented with additional confidentiality agreements. The lab must therefore operate with some confidentiality firewalls inside. Lab members who are inside formal standards efforts too will often not be able to report back, to the original authors of contributions, any details about how these contributions are being handled further inside their formal standards efforts.
Lab members
Current lab leadership
Co-project lead: Ariel Gil
Co-project lead: Jonathan Happel
Resident standards writing and standards process expert: Koen Holtman
Past and present contributing volunteers
Not all contributing volunteers are listed below. Volunteers can choose to not to have their names, or their full names, disclosed in public.
Christopher Denq
Jonathan Claybrough