AI Procurement in the Public Sector: What Queensland Agencies Need to Know

Introduction

Queensland government agencies are under increasing pressure to adopt artificial intelligence — to improve service delivery, manage growing demand, reduce administrative burden, and make better use of data. At the same time, public sector AI procurement carries obligations and constraints that do not apply in the private sector: procurement rules, ministerial accountability, privacy legislation, ethical frameworks, and the requirement to treat all citizens fairly. Getting this right is genuinely complex. Agencies that rush AI procurement to meet political timelines risk expensive failures, reputational damage, and harm to the people they serve. Agencies that are too cautious risk falling behind and failing to deliver the productivity improvements that would allow them to serve the public better. This article provides practical guidance for Queensland public sector leaders responsible for AI procurement decisions.

The Queensland Procurement Policy and AI

AI procurement in Queensland agencies must comply with the Queensland Procurement Policy (QPP), which requires value for money, open and effective competition, support for local industry, ethical behaviour, and accountability. These requirements interact with AI procurement in specific ways that are worth understanding before you go to market. The value for money requirement means that lowest cost is not the right criterion for AI procurement. A cheaper model that produces biased outputs, requires extensive customisation, or cannot be audited is not value for money. Evaluation criteria should weight accuracy, explainability, vendor support, data sovereignty, and long-term total cost of ownership. The ethical behaviour requirement has particular relevance for AI. Queensland agencies are expected to act ethically and to ensure their suppliers do the same. Procuring AI systems that cannot explain their decisions, that use data in ways citizens have not consented to, or that produce discriminatory outcomes creates ethical and legal exposure for the agency and its responsible officers.

The Australian Government's Voluntary AI Safety Standard

In 2024 the Australian Government published its Voluntary AI Safety Standard, which provides a framework for responsible AI development and deployment. While voluntary for the private sector, Queensland agencies should treat its ten guardrails as a practical minimum standard for AI procurement and deployment. The guardrails cover accountability, risk management, transparency, testing, human oversight, explainability, privacy, security, contestability, and incident reporting. When evaluating AI vendors, agencies can use these guardrails as evaluation criteria — asking vendors to demonstrate how their system meets each one. Agencies that procure AI systems that demonstrably meet the Voluntary AI Safety Standard are in a significantly stronger position when questioned by ministers, auditors, or the public about how a decision was made.

Data Sovereignty and the Queensland Context

Data sovereignty is a critical consideration for Queensland public sector AI procurement that is often underweighted. Citizens who interact with Queensland government services have a reasonable expectation that their data is held in Australia, subject to Australian law, and not accessible to foreign governments through foreign cloud infrastructure legislation. This means that AI solutions hosted offshore — or using models trained or fine-tuned on data held offshore — require careful legal assessment. The Information Privacy Act 2009 (Qld) restricts the transfer of personal information outside Australia except in specific circumstances. The Queensland Government Information Security Classification Framework imposes additional requirements on systems handling sensitive or protected information. When evaluating cloud-based AI solutions, agencies should require vendors to specify precisely where data is stored, processed, and transmitted. "We use AWS" is not sufficient — you need to know whether data stays in the ap-southeast-2 (Sydney) region and never leaves it, and whether the vendor's own staff outside Australia can access it.

Algorithmic Accountability in Public Services

When a Queensland government agency uses AI to make or significantly influence decisions about individuals — benefit eligibility, child safety risk assessments, licence applications, compliance decisions — there are accountability obligations that go beyond the technology itself. The decision-maker in Queensland administrative law is the officer, not the algorithm. If an AI system recommends a decision and an officer approves it without genuine scrutiny, the accountability framework may not protect the officer or the agency if that decision is later challenged. Agencies need governance frameworks that distinguish between AI as a tool that informs human judgment and AI as a replacement for it. The Queensland Ombudsman and the Queensland Human Rights Commission have both indicated interest in how agencies are using automated systems. Agencies that cannot demonstrate human oversight, audit trails, and contestability mechanisms are exposed.

Evaluation Criteria for Public Sector AI

Effective AI procurement evaluation criteria for Queensland agencies should cover five areas beyond cost and functionality. The first is explainability — can the system explain its outputs in plain language, and can those explanations be provided to citizens and reviewed by human officers? Black-box AI is increasingly difficult to defend in a public sector context. The second is bias and fairness testing — has the vendor tested the system for discriminatory outputs across relevant demographic groups, and what is their process for ongoing monitoring? AI systems trained on historical data can encode historical inequities. The third is data governance — who has access to the data used to train and operate the system, where is it stored, and what happens to it if the contract ends? The fourth is auditability — can the system produce logs sufficient to reconstruct any decision for review by an auditor, ombudsman, or court? The fifth is vendor stability and exit — what is the vendor's track record, and what is the exit pathway if the vendor is acquired, fails, or significantly changes their pricing?

Building Internal Capability Alongside Procurement

A recurring failure pattern in public sector AI is procuring a sophisticated system that the agency does not have the internal capability to manage, oversee, or challenge. Agencies that rely entirely on vendors to tell them whether their AI is working correctly have surrendered accountability they cannot afford to surrender. Every significant AI procurement should be accompanied by a capability investment — training for the officers who will use the system, technical literacy for the managers who will oversee it, and at least one internal person who understands the system well enough to ask hard questions of the vendor.

Related reading:

• the 2024 Privacy Act amendments
• AI and data privacy in Australia
• managing risk in AI-driven systems
• choosing the right AI solution

Conclusion

AI procurement in the Queensland public sector requires balancing innovation with accountability in a way that the private sector does not. The stakes are different: the people affected by public sector AI decisions often have no alternative provider to turn to, and the agencies making those decisions are accountable to ministers, auditors, and the public in ways that private organisations are not. Getting it right is possible, and the agencies that build strong foundations — clear governance, sound procurement criteria, genuine internal capability — will be able to move faster and with more confidence than those that rush in without them. If your agency is assessing its AI readiness before a procurement decision, contact us to discuss your specific situation.