Recent years have seen an explosion in interest in artificial intelligence and big data. The technology is promising, but its use in government rightly makes people nervous.
Subscribe now for unlimited access.
or signup to continue reading
Yet it's useful to see areas where artificial intelligence and big data have helped produce better outcomes for citizens, without undermining key ethical values of transparency, privacy and human oversight.
Australian Bureau of Statistics analysts are at the forefront of methodological innovation, helping to improve our understanding of the Australian economy and labour market.
Take the example of the Australian Bureau of Statistics using generative artificial intelligence to help update the Australian and New Zealand Standard Classification of Occupations.
The Australian and New Zealand Standard Classification of Occupations is a vital data set for understanding the nature of the labour market. It catalogues and categorises all occupations based on skills in Australia and New Zealand. Last updated in 2006, it now covers 1076 occupations, from acupuncturists, blacksmiths and cartographers to wool classers, youth workers and zookeepers.
Over time, occupations change. Bringing a 2006 classification up to date in 2024 was set to be a significant task. Just think about making a discrete list of the key tasks in your job and multiply that by more than a thousand jobs across the economy.
That's why the Australian Bureau of Statistics looked to artificial intelligence for help to create a preliminary list of tasks undertaken in each occupation.
Australian Bureau of Statistics data scientists gave ChatGPT a comprehensive prompt of over 480 words based on the existing publicly available occupational classification. This helped the artificial intelligence to generate its output in the right format and style.
Australian Bureau of Statistics analysts spent time testing and refining "the prompt" - or the question they ask ChatGPT - so that the machine delivered what the humans needed.
After each test, the analysts used a mathematical formula to calculate the quality of the responses within tolerance levels around precision and recall.
The Australian Bureau of Statistics data scientists iterated the prompt until it consistently produced a high-quality output. The final prompt ended up optimising the output from ChatGPT to about 69 per cent for both precision and recall.
The results were of such high quality that in one test, experts themselves were asked to distinguish between the description of occupational tasks written by experts and those written by generative artificial intelligence. Two-thirds got it wrong.
The results from ChatGPT were not perfect. But they did provide enough of a starting point for Australian Bureau of Statistics analysts to review and build on. As a result, the project team saved approximately 1600 hours. It amounted to an approximate seven-fold return on investment.
Throughout, Australian Bureau of Statistics data scientists were clear about the purpose of the exercise - to use generative artificial intelligence to support - not replace - human analysts. All outputs were evaluated against four criteria: quality; ethics; legality; and security. This meant that artificial intelligence outputs were correct, maintained privacy, did not break intellectual property laws and vitally, maintained the place of humans at the centre of decision-making.
This delivers on the Australian government's commitment to fostering an innovative culture in the Australian Public Service, while managing the risks of emerging technologies.
This low-risk project is a promising example of how generative artificial intelligence can be applied in national statistics, enhance productivity, and develop related skills and capabilities in safe and responsible ways. It will help to pave the way for further Australian Bureau of Statistics' adoption of artificial intelligence technologies.
The Australian Bureau of Statistics is also at the forefront of harnessing digital technologies to build high quality, big data sets. By collecting weekly supermarket scanner data from private companies, the Australian Bureau of Statistics is able to gather granular insights about household inflation and understand how the economy is tracking.
This is big data. For a moment, close your eyes and try to imagine all the items in Australia which were scanned at major supermarket checkouts over the past week. If it helps, remember that the typical major supermarket stocks over 15,000 different products, that there are over 10 million households in Australia, and that the typical household shops more than once a week.
Well, each of those items scanned last week - including the product purchased, quantity purchased, dollar value and location of purchase - is logged in this big data set. The Australian Bureau of Statistics has been collecting this weekly data since 2011, so you can only imagine the size of it.
Through this data provision arrangement between private business and the Australian Bureau of Statistics, 84 per cent of food sales from supermarkets and grocery stores is captured. The scale of this data allows for a granular understanding of household consumption, which is essential for compiling estimates of the consumer price index.
The Australian Bureau of Statistics is continuing its work to modernise collection methods to gather high quality and granular household spending data.
Traditionally, the Australian Bureau of Statistics has collected detailed data by sending survey forms to households and asking them to record every good and service they purchase in a two-week period (using a 28-page physical diary). This process is expensive, and response rates are falling. The next expenditure survey will utilise a digital diary - much like an app - which will work on any electronic device. This will greatly reduce costs and burden as well as streamline data processing.
The current revolutions in artificial intelligence and big data in national statistics are not simply good just because they use new, cutting-edge technologies. They matter because they offer the Australian government a way of improving our administrative practices, and therefore the way we deliver for all citizens.
Artificial intelligence and big data can help to structure and collect data productively, safely and responsibly.
Big data and generative artificial intelligence offer opportunities across government. The Australian government remains committed to exploring these technologies within a clear ethical framework. Better data allows us to better serve the nation.
- Andrew Leigh is the Assistant Minister for Treasury, and his website is andrewleigh.com.
