The Act follows the information, not the product name
A New Zealand private-sector business is a New Zealand agency under the Privacy Act 2020 when it is established under New Zealand law, or when its central management and control is in New Zealand1. Councils and government departments are agencies too. The Office of the Privacy Commissioner puts the point in operational language: the Act applies to everyone using AI tools in New Zealand, and the information privacy principles are the legal test for collection, use and sharing2. Copilot is one of those tools. Buying it from a familiar vendor does not move the duty onto Microsoft.
MBIE's responsible AI guidance for businesses makes the same link. It treats the Privacy Act 2020, and any applicable code, as a duty when personal information is in an AI input or output, and it says a business that handles personal information must appoint a privacy officer4. Section 201 of the Act is that duty1. The officer encourages compliance, handles access and correction, and works with the Commissioner1. A sole trader holding information only for personal or domestic affairs is the exception. A firm with staff or clients is not.
Which principles show up in a Copilot prompt
The principles are set out in section 22 of the Act1. Collection is covered by principles 1 to 4, and, since 1 May 2026, by principle 3A as well1. Use, security and retention sit in principles 5 to 101. Disclosure is principle 11, overseas disclosure is principle 12, and unique identifiers are principle 131. The Commissioner's guide on AI and the principles, published in September 2023, still groups the principles as collection, use and protection, and sharing2. Read the Act for the current text, including principle 3A, and use the Commissioner's guide for the questions to ask2.
Principle 1 allows collection only for a lawful purpose connected with the agency's work, and only as far as that purpose needs1. Principle 10 then limits use1. Information obtained for one purpose is not to be used for another unless a listed ground applies, such as a directly related purpose or the person's authorisation1. A file given so you can do the job is not, by itself, permission to experiment with a new feature.
Principle 8 of the Act is the one staff feel every day1. An agency must not use or disclose personal information without taking reasonable steps to check that it is accurate, up to date, complete, relevant and not misleading1. The Commissioner warns that generative tools produce confident errors and can repeat bias, and expects a human to review output before the agency acts on it3. In Copilot that means the draft email, the meeting note and the spreadsheet insight are not finished work. Someone who knows the file has to read them against the source.
Principles 6 and 7 are the access and correction rights1. If Copilot output is stored back into a mailbox, a document library or a case note, it can become personal information the business holds. The person concerned can ask for it and can ask for a correction1. The Commissioner says generative tools are not always compatible with those rights, and expects a procedure for access and correction where the tool collects personal information about customers or clients3. Decide, before the pilot, which store is the record and how a correction gets into it.
Security, prompts and a notifiable breach
Principle 5 requires reasonable security safeguards against loss, unauthorised access, use, modification or disclosure, and other misuse1. If information is given to someone else so they can provide a service, the agency must do what is reasonably in its power to stop unauthorised use or disclosure1. A cloud tool is that kind of service. The safeguard is not a hopeful reading of a marketing page. It is the contract, the admin settings, and the rule about what staff may type.
Microsoft's enterprise data protection page describes the commitments that apply to prompts and responses in Microsoft Copilot and Microsoft Copilot Chat for organisations. Microsoft acts as a processor under the Data Protection Addendum and the Product Terms. Prompts, responses, and material reached through Microsoft Graph are not used to train foundation models. Copilot honours the tenant's permissions, sensitivity labels, retention and audit settings, to the extent the subscription includes them5. Microsoft's architecture page adds that customer data stays inside the Microsoft 365 service boundary, and that Copilot cannot open material the signed-in person is not allowed to open6. Those are meaningful controls. They do not replace principle 5 of the Act1. Over-shared sites are still inside the boundary.
A privacy breach includes unauthorised or accidental access to, or disclosure, alteration, loss or destruction of, personal information1. A notifiable privacy breach is one that it is reasonable to believe has caused serious harm, or is likely to1. Section 113 of the Act lists factors the agency must consider, including actions taken to reduce the risk1. Section 114 requires notice to the Commissioner as soon as practicable after the agency becomes aware of a notifiable breach1. Section 115 requires notice to the affected person, or public notice if individual notice is not reasonably practicable, subject to the exceptions in the Act1. The Commissioner points agencies to the NotifyUs tool for that report3.
Stats NZ's 2022 Business Operations Survey, covering businesses with six or more employees, found that 6 percent of those with an internet connection reported a damaging ICT security incident in the previous two financial years7. The National Cyber Security Centre says the same network controls, including multi-factor authentication and a rule for what may be typed, should cover the AI system8.
Overseas disclosure is a separate question from security
Principle 11 of the Act limits disclosure1. Principle 12 adds a test when the recipient is outside New Zealand1. Several of the principle 11 grounds are available only if a principle 12 condition is also met: informed authorisation, a belief the recipient is subject to the Act, comparable privacy laws, a prescribed scheme or country, or a contract that requires comparable protection1. Section 23 of the Act, which covers an action on information held overseas that a foreign law requires, is narrow1. It is not a reason to skip the test.
For Copilot, write down the service, where the relevant data rests, and the contract clause you rely on. Microsoft lists New Zealand as a Local Region Geography and lists Auckland as a data-centre location. The standard Product Terms commitment for Copilot data at rest names a list of countries and regions, and New Zealand is not on it. Advanced Data Residency is the add-on that can commit Copilot and Copilot Chat data to a Local Region Geography, including New Zealand9. Save the Data Location Card with the privacy impact assessment.
Web search is a different disclosure. Microsoft says a web query is a few words, sent without user or tenant identifiers, and not used to train foundation models. The Data Protection Addendum does not apply. Bing sits under the Microsoft Services Agreement, with Microsoft as an independent controller10. Block web search for teams that handle client or staff information unless the assessment accepts that risk.
What the Commissioner expects before the switch is flipped
The Commissioner first set out expectations for agencies considering a generative tool on 25 May 2023, and the update of 15 June 2023 lists eight of them3. Senior leaders approve the use after looking at risks and mitigations. The agency asks whether the tool is necessary and proportionate. It completes a privacy impact assessment, including feedback from affected groups, and it looks at what the provider has published about privacy. People are told, in plain language, when the use is likely to affect them. The agency engages with Māori about impacts on communities and on information that is a taonga. There is a procedure for accuracy and for access. A person reviews output before the agency acts. Personal or confidential information is not entered unless the provider has confirmed it is not retained or disclosed3.
The later guide asks whether the training data is reliable, whether the use matches the purpose of collection, and how prompts will be kept secure. If you are in doubt, the Commissioner's recommendation is not to use an AI tool on personal information2. The Commissioner does not endorse vendors3. Read the enterprise data protection commitments, check the tenant, and record the decision for a named set of tasks5.
Public-service guidance is a benchmark, not a second statute
The Government Chief Digital Officer's responsible AI guidance for the public service says the Privacy Act 2020 applies to generative AI, and that agencies should use privacy impact assessments for testing and for live use. Its plain example is the one private firms should copy for anything they would not want published: do not enter personal information into a public generative system. Public information, or a request for an empty report structure, is a different matter. The guidance also points back to the Commissioner's material11.
A Microsoft 365 tenant with enterprise data protection is not the same thing as a public chatbot tab. The GCDO example is still the right test for a personal login. MBIE tells businesses to classify information so they know what is personal, and to line AI use up with the security and privacy rules they already have4. Record Microsoft's training answer against the Copilot offer you licensed. Microsoft separates web search from the processor terms, and it says to read an agent's own privacy statement before that agent sees client content5,10.
A sequence a privacy officer can run
Start with an inventory, not a licence count. List the teams that want Copilot, the kinds of personal information they handle, and the SharePoint sites and mailboxes they already use. The Commissioner's collection principles, including the new duty in principle 3A to tell people when you have collected information about them from someone else, matter if Copilot will be used to assemble a note about a person from files they did not hand you in that moment1. If the use is only to tidy a document the person already knows you hold, say that. If the use will create a new profile, stop and redesign it.
Second, match each use to a Microsoft control you have actually turned on. Permissions, labels, retention, audit, conditional access and multi-factor authentication are the list on Microsoft's architecture and enterprise data protection pages5,6. The NCSC expects multi-factor authentication on the AI system and on stores of training data8. Turn web search off for the pilot group unless the assessment allows it10. Save a copy of the Data Location Card with the note on principle 129.
Third, tell people. Clients and staff should hear, in plain language, about a use that affects them3. Tell staff what must not be pasted into an unapproved tab.
Finally, decide what would make you turn the pilot off. A label that does not stick, a site shared with the whole company, a web-search query that included a client identifier, or an output filed as fact without a check, are all reasons to pause. Principle 5 and the notifiable-breach provisions are easier to meet when the pause is planned1. The rollout can resume when the control is fixed. It should not resume because the calendar said the training week had arrived.