Duplicates: the tax you did not know you were paying
Every M365 tenant hoards duplicates. The average enterprise we scan has 8-15% of its SharePoint bytes tied up in exact copies of the same file. Here is why, and what to do about it.
Duplicates in M365 are not a moral failing of your users. They are a rational response to how people use SharePoint — attach the same PDF to a Teams chat, save it to OneDrive to sign it, then upload it to the client site. Three copies. Same file. Nobody did anything wrong.
How TSO finds them
TSO computes a content hash for every file below the workload storage threshold (tunable per tenant) and groups files by identical hash. Two files with the same hash are byte-identical — no false positives from renamed copies or slightly-different metadata.
For each duplicate group we report:
- How many copies exist across the tenant.
- How many bytes you would reclaim if all but one were removed.
- Which copy is the earliest — usually the source of truth.
- Which sites and OneDrives hold the copies, so you can see if the duplication is systemic or user-specific.
What to do about them
The right action varies by duplicate class:
- Attachments hoarded in email — set an Exchange retention policy on the shared mailboxes that hold them; the emails go, the SharePoint originals stay.
- PDFs re-uploaded from OneDrive to a client site — leave the client copy, cull the OneDrive one after conversation with the owner.
- System-generated duplicates (Teams recording copies, Word .asd temp files, Office temp lock files) — Purview retention labels handle these at scale.
Do not blanket-delete. TSO ranks by reclaim value so you can spend time on the top-N groups; the long tail rarely justifies the review effort.