Researchers at Wangxuan Institute of Computer Technology at Peking University and the Computer Science Department of the University of California (Los Angeles) developed the Quantitative Reasoning with Data (QRData) Benchmark to assess LLM's ability to analyze statistical data. QRData includes data from open texts books, research papers, and other sources and is combined with 411 questions. Of the LLM's tested, GPT-4 performed the best, but the researchers noted the need for improvement.
Observatory of Examples of How Open Data and Generative AI Intersect
A growing observatory of examples of how open data from official sources and generative artificial intelligence (AI) are intersecting across domains and geographies.
Share your project for inclusion. We seek to learn from generative AI initiatives that use open government and research data across a Spectrum of Scenarios. More information on each scenario can be found in our report: A Fourth Wave of Open Data? Exploring the Spectrum of Scenarios for Open Data and Generative AI.