[…]

Assembly Committee on Privacy and Consumer Protection — Analysis of SB 53 (Wiener)

Committee: Assembly Committee on Privacy and Consumer Protection Bill: SB 53 (Wiener) — Artificial Intelligence Models: Large Developers Hearing Date: July 8, 2025 Analysis Prepared by: John Bennett / P. & C.P. / (916) 319-2200

SB 53 Date of Hearing: July 16, 2025 Fiscal: Yes

ASSEMBLY COMMITTEE ON PRIVACY AND CONSUMER PROTECTION Rebecca Bauer-Kahan, Chair SB 53 (Wiener) – As Amended July 8, 2025

SENATE VOTE: 37-0

PROPOSED AMENDMENTS

SUBJECT: CalCompute: foundation models: whistleblowers

[…]

17. Establishes within the Government Operations Agency a consortium to develop a framework for the creation of a public cloud computing cluster to be known as “CalCompute.”
18. Requires the consortium to develop a framework for the creation of CalCompute that advances the development and deployment of artificial intelligence that is safe, ethical, equitable, and sustainable by doing, at a minimum, both of the following:

• a. Fostering research and innovation that benefits the public.
• b. Enabling equitable innovation by expanding access to computational resources.

19. Requires that the consortium make reasonable efforts to ensure that CalCompute is established within the University of California to the extent possible.
20. Requires CalCompute to include, but not be limited to, all of the following:

• a. A fully owned and hosted cloud platform.
• b. Necessary human expertise to operate and maintain the platform.
• c. Necessary human expertise to support, train, and facilitate the use of CalCompute.

21. Requires the consortium to operate in accordance with all relevant labor and workforce laws and standards.
22. Requires Government Operations Agency to submit a report from the consortium to the Legislature with the framework developed by this bill for the creation and operation of CalCompute on or before January 1, 2027.
23. Requires the report required by 22 to include all of the following elements:

• a. A landscape analysis of California’s current public, private, and nonprofit cloud computing platform infrastructure.
• b. An analysis of the cost to the state to build and maintain CalCompute and recommendations for potential funding sources.
• c. Recommendations for the governance structure and ongoing operation of CalCompute.
• d. Recommendations for the parameters for use of CalCompute, including, but not limited to, a process for determining which users and projects will be supported by CalCompute.
• e. An analysis of the state’s technology workforce and recommendations for equitable pathways to strengthen the workforce, including the role of CalCompute.
• f. A detailed description of any proposed partnerships, contracts, or licensing agreements with nongovernmental entities, including, but not limited to, technology-based companies, that demonstrates compliance with the requirements of 19 and 20.
• g. Recommendations regarding how the creation and ongoing management of CalCompute can prioritize the use of the current public sector workforce.

24. Requires that the consortium to consist of 14 members as follows:

• a. Four representatives of the University of California and other public and private academic research institutions and national laboratories appointed by the Secretary of Government Operations.
• b. Three representatives of impacted workforce labor organizations appointed by the Speaker of the Assembly.
• c. Three representatives of stakeholder groups with relevant expertise and experience, including, but not limited to, ethicists, consumer rights advocates, and other public interest advocates appointed by the Senate Rules Committee.
• d. Four experts in technology and artificial intelligence to provide technical assistance appointed by the Secretary of Government Operations.

25. States that the members of the consortium serve without compensation, but will be reimbursed for all necessary expenses actually incurred in the performance of their duties.
26. Requires the consortium to be dissolved upon submission of the report to the Legislature.
27. Permits the University of California may receive private donations for the purposes of implementing CalCompute if CalCompute is established within the University of California.
28. Requires that CalCompute become operative only upon an appropriation.

[…]

The recently released International AI Safety Report, developed by nearly 100 internationally recognized experts from 30 countries led by Turing Award winner Yoshua Bengio, sets forth three general risk categories associated with frontier models: malicious use, malfunctions, and systemic risk.

[…]

• Systemic risks arise from widespread deployment and reliance on foundation models. Such risks include labor market disruption, global AI research and development concentration, market concentration, single points of failure, environmental risks, privacy risks, and copyright infringement.[^12 ]

[…]

Systemic risks. Due to the high costs of developing AI systems, a small number of large technology companies dominate the frontier model market, compounding many of the risks described above. Widespread use of a few frontier models can make critical sectors such as healthcare and finance vulnerable to systemic failures if a model has flaws, vulnerabilities, bugs, or biases.[^33] Additionally, “[t]hose in control of powerful systems may use them to suppress dissent, spread propaganda and disinformation, and otherwise advance their goals, which may be contrary to public wellbeing.”34 The potential implications for, among other issues, labor displacement, inequality, democracy, and human rights are profound.

[…]

Beyond the Board of Frontier Models and the bill’s safety and transparency provisions, SB 1047 would have required computing clusters to implement procedures to evaluate whether customers intended to use their infrastructure to train a covered model. The bill also would have established, within GovOps, a consortium tasked with developing a framework for a public cloud computing cluster, CalCompute, to support the safe development and deployment of AI. SB 1047 also included whistleblower protections, allowing employees to report noncompliance to either the Labor Commissioner or the AG.

[…]

CalCompute. This bill, similarly to SB 1047, also establishes a consortium within the GovOps to develop a framework for creating a public cloud computing cluster known as “CalCompute.” This initiative responds to the fact that academic institutions currently lack sufficient computing power to conduct research at the scale of large developers. This creates a resource and research gap, where the academic institutions, typically responsible for studying the safe and effective use of new technologies, are unable to keep pace with advancements at the AI frontier. AI has the potential to transform our economy and power new industries; however, this transformation can only be fully and equitably realized with public support. The establishment of CalCompute aims to advance that goal by ensuring academic institutions have the necessary resources to conduct essential research on foundation models that will inform and protect the public. Specifically, the consortium will develop a framework for CalCompute that promotes the safe, ethical, equitable, and sustainable use of AI. The framework development will include a report analyzing the state’s current cloud computing infrastructure, the costs of building and maintaining CalCompute, and the state’s technology workforce. The report will also offer recommendations for equitable pathways to strengthen the workforce and outline CalCompute’s role in supporting these efforts. Furthermore, the report must include recommendations for CalCompute’s governance and operation, usage parameters, and how its creation and ongoing management can prioritize the employment of the current public sector workforce. The bill requires CalCompute to feature a fully owned and hosted cloud platform, staffed with the necessary human expertise to operate, maintain, support, train, and facilitate its use. The consortium must prioritize locating CalCompute within the University of California system. If established there, CalCompute may also accept private donations. The consortium will consist of four representatives from the University of California and other public and private academic research institutions and national laboratories, along with four technology and AI experts appointed by the Secretary of Government Operations to provide technical assistance. The Speaker of the Assembly will appoint three representatives from impacted workforce labor organizations, and the Senate Rules Committee will appoint three representatives from stakeholder groups with relevant expertise and experience.

[…]

Specific attention was also focused on third-party evaluations (sometimes referred to as audits in the feedback) by highlighting the immature third-party evaluation ecosystem, the lack of evaluation standards and broader measurement science, and the current barriers for evaluators to acquire access to the AI models and systems they evaluate.[^49]

[…]

Moreover, a core function of government, is to craft sound industrial policy that encourages the growth of new markets serving the public good when economic forces do not support that market. In this case, fostering a robust AI auditing ecosystem capable of verifying compliance with developers’ SSPs and evaluating whether appropriate methodologies are being used to mitigate catastrophic risks is in the public interest.

[…]

ARGUMENTS IN SUPPORT**: Secure AI Project, co-sponsors of the bill, alongside a coation of technology equity advocacy groups write in support:

[…]

Finally, SB 53 would also set in motion CalCompute, a public cloud computing cluster for use by academics and startups in California. Computational resources are essential for AI research and CalCompute would make those resources more accessible to California’s top universities and startups, helping to catalyze additional research into beneficial applications of AI and supporting, in particular, smaller startups for a healthier innovation ecosystem. This mirrors a similar computing cluster that is already being established in New York state. We support this groundbreaking effort, which would advance and democratize AI research in California. SB 53 thoughtfully implements the recommendations of the Report by combining a low-burden transparency and reporting regime with a public compute cluster that will broaden access for AI researchers and startups in California. This is a commonsense approach that will strengthen the AI ecosystem, benefiting both companies and the public interest.

[…]

REGISTERED SUPPORT / OPPOSITION

Support

[…]

• University of California

[…]