Document Type
Article
Publication Date
2-2024
Abstract
We performed a billion locality sensitive hash comparisons between artificially generated data samples to answer the critical ques- tion - can we reproduce the results of generative AI models? Repro- ducibility is one of the pillars of scientific research for verifiability, bench- marking, trust, and transparency. Futhermore, we take this research to the next level by verifying the “correctness” of generative AI output in a non-deterministic, trustless, decentralized network. We generate mil- lions of data samples from a variety of open source diffusion and large language models and describe the procedures and trade-offs between gen- erating more verses less deterministic output. Additionally, we analyze the outputs to provide empirical evidence of different parameterizations of tolerance and error bounds for verification. For our results, we show that with a majority vote between three independent verifiers, we can detect image generated perceptual collisions in generated AI with over 99.89% probability and less than 0.0267% chance of intra-class collision. For large language models (LLMs), we are able to gain 100% consen- sus using greedy methods or n-way beam searches to generate consensus demonstrated on different LLMs. In the context of generative AI train- ing, we pinpoint and minimize the major sources of stochasticity and present gossip and synchronization training techniques for verifiability. Thus, this work provides a practical, solid foundation for AI verification, reproducibility, and consensus for generative AI applications.
Creative Commons License
This work is licensed under a Creative Commons Attribution 4.0 International License.
Rights
Licensed to Smith College and distributed CC-BY 4.0 under the Smith College Faculty Open Access Policy.
Recommended Citation
Kim, Edward; Isozaki, samu; Sirkin, Naomi; and Robson, Michael, "Generative Artificial Intelligence Consensus in a Trustless Network" (2024). Computer Science: Faculty Publications, Smith College, Northampton, MA.
https://scholarworks.smith.edu/csc_facpubs/423
Comments
Author’s submitted manuscript.