Gokan's Studio

Governance Gaps, AI Limits, and Missed Opportunities by Gokan Ozcifci, Joanne C Klein, Drew Madelung, Mike Maadarani, and Steve Corey

In early 2024, Microsoft introduced the ability to autofill metadata in a SharePoint library. This feature allows for automatic extraction, summarization, or generation of content from files uploaded to a SharePoint document library.

A bit after a year, on September 18, 2025, Microsoft launched the SharePoint Knowledge Agent in public preview. This AI-powered assistant enhances SharePoint content by adding structure and metadata, making information more accessible and helpful when working with Microsoft 365 Copilot. The Knowledge Agent helps organizations organize, enrich, and streamline content for more innovative collaboration with Autofill in the backend. You can use it with either the Knowledge Agent or PAYG.

If you have never heard of or added any Autofill column, you can always check this article: Create an autofill column | Microsoft Learn

Today, if you browse the web, check forums, or even skim through social media discussions, it seems like everyone is singing praises about Autofill in SharePoint. “Autofill is the next-gen-ai tool for metadata”, “Autofill is the solution for content generation”, “Autofill solves everything”—you name it. Everywhere you look, it’s all sunshine and rainbows.

But here’s the thing: reality doesn’t always match the hype. While Autofill brings some impressive capabilities, it’s not flawless. Things don’t always work as expected, certain limitations can catch you off guard, and some scenarios reveal cracks in the system. We are MVPs that tell the truth, not free marketeers 😊

That’s why I decided to write this article based on world-renowned MVPs within the SharePoint AI era. My goal is simple: to take a step back from the fanfare and address some key drawbacks, missed opportunities, and considerations about Autofill that rarely get discussed. Because yes, it’s excellent—but it’s also essential to know where it can trip you up.

Ready?

Let’s go!

1# Content Type and Site Column SupporT.

• Autofill is configured at the library level and is not integrated with Content Types and Site Columns.
• At scale, enterprises rely on a centralized Content Type Gallery, Site Columns, and hub-level provisioning (PnP, templates, automation) for consistency and governance.
• Without native bindings to content types/site columns, deployment is brittle and one-off, leading to drift and rework.

Enterprise Impact?

• Inconsistent metadata across thousands of libraries undermines search, retention, and sensitivity labeling.
• Higher admin overhead to replicate settings manually.
• Governance gaps (IA drift) that create audit findings in regulated sectors (FSI, Life Sciences, Public Sector).

What would be nice?

• Content Type–aware Autofill
  • Allow configuration of Autofill on a Content Type so that any library using that Content Type inherits the Autofill mappings.
• Site Column mapping & reuse
  • Bind extracted fields to Site Columns (not just list columns).
• Hub/Tenant-scale deployment
  • Provide templateable Autofill configurations (SP site script/site design & PnP) and Graph/REST APIs to automate rollout across hubs/portfolios.
  • Include versioning of Autofill configurations with roll-forward/rollback.
• Policy & Governance
  • Site-level policy to restrict Autofill to approved content types/site columns.

2# Trust & ModelS.

In many document types (contracts, clinical protocols, policies, RFPs), wrong metadata creates real risk (e.g., retention errors, mis-filed contracts, misapplied sensitivity). Today, users lack explanations, confidence scoring, validation rules, and human-in-the-loop gates to trust AI outputs like Autofill reliably.

Enterprise Impact?

• Business owners resist enabling Autofill where it would help most.
• Compliance teams block adoption without auditability and controls.
• The promise of “AI-assisted governance” remains out of reach.

What would be nice?

• Per-field confidence scores with why/explain overlays (snippet-level evidence, field provenance).
• Highlight the source text region for each extracted value (click to view in file preview).
• Out-of-distribution detection when a document looks unlike others in the library.

Validation & Policies

• Thresholds per field/column: below threshold → mark “Needs Review”.
• Schema rules: regex for IDs, date ranges, lookup validation for Managed Metadata.
• Dual-source extraction: AI + rules (pattern/regex/classifier) with tie-break policies.

Human-in-the-loop

• Review queues with bulk triage: accept/correct/explain why wrong.
• Learning loop: accepted/corrected items feed back into Autofill prompt.

Prompt Lifecycle & Audit

• Versioned prompts: tied to extraction runs; store which version populated which values.
• Complete audit trail: who/what/when changed metadata, including AI vs. human source.

3# Save as a TemplatE.

One of the current limitations of Autofill is that it only operates within a single document library per site. If you manage multiple sites—say, ten different SharePoint sites—you end up having to copy and paste the same prompt across every library. Sure, that’s how Autofill is designed to work, but it quickly becomes repetitive and time-consuming. When an enterprise has dozens (or hundreds) of sites, each site’s Knowledge Agent suggestions may diverge, leading to a significant invisible mess.

Because the Knowledge Agent can suggest or autofill metadata columns, different parts of your SharePoint estate might accumulate many metadata fields that are semantically similar (but not the same)—for example: “ClientName,” “Customer Name,” “Client Name,” “customer_name,” etc. Some will be free text, others choice fields, and perhaps some will use managed metadata, while others will be simple text. That leads to:

• Duplication: Multiple fields that are conceptually the same but technically distinct.
• Confusion: Users or downstream tools may not know which metadata field is “right” to use.
• Fragmentation: Search filters, reports, dashboards, and processes that depend on metadata suffer when similar concepts are scattered across many fields with different names or types.

Consequences

• Reduced reliability of automated processes that use metadata (e.g. compliance, retention, reporting).
• Search & retrieval returning incomplete or unexpected results because some similar documents are tagged under “ClientName,” others “Customer Name.”
• Maintenance burden: having to clean up or map fields after the fact, or manage dozens of redundant metadata fields.

4# Conditional FormattiNG.

What’s really missing is the ability to incorporate conditional formatting directly into your prompts. Imagine if you could define rules once—like “if the status is overdue, mark it red; if complete, mark it green”—and have that logic applied automatically wherever Autofill runs. This would combine your prompt engineering with intelligent formatting in a single step, saving hours of repetitive work.

A practical example: your manager decides that a certain status should now be red instead of green. Today, you would need to manually update this in every document library across all your sites. With conditional formatting built into Autofill and with the “save as a template”, a single change to your rule would propagate everywhere—one tweak, zero repetition, maximum efficiency.

It’s clear that adding conditional formatting isn’t just a nice-to-have—it’s a real productivity booster that would make Autofill truly more innovative and more flexible.

Bear in mind that my prompt is working well—like, replacing the $ sign with the € sign—but once a conditional access is added, it fails miserably, unfortunately.

5# Weak Audibility, Version Control, AND Undo-Ability.

Because the Agent sometimes autofills or changes metadata, or suggests new columns, changes may be made that are hard to trace, reverse, or determine exactly who accepted what and when. Specifics:

• No detailed audit trail for acceptance/rejection of metadata suggestions.
• Autofill operations may alter metadata on many documents, sometimes in bulk, with no “undo” or rollback to prior versions.
• When columns are added via Agent suggestions, decisions may be made (or accepted) without coordination or quality checking.

People tend to focus on the improvements (search, less manual work), but don’t often consider how to manage mistakes or temporal drift. Users may assume that metadata suggestions are “safe,” but when errors accumulate, remediation becomes painful. Also, because the preview versions often lack robust logs, many enterprise adopters may be blindsided.

Consequences

• Mis tagged or incorrect metadata persists, potentially leading to misinformation or misclassification.
• Difficulty in meeting compliance, audit, or legal requirements if you can’t show who applied metadata or when.
• User distrust: once users see wrong metadata or stale fields, they may ignore or override them, defeating the purpose.

6# Language and Silent HallucinatioN.

Language Detection is impressive, but translation is missing. Autofill, in general, supports the same languages as Copilot. As you can see in the image below, my Turkish prompt was perfectly understood and processed. However, the document translation service has existed for a long time, and I think it’s a missed opportunity not—or not give the option— to translate the extracted values | Overview of document translation | Microsoft Learn.

Hallucination is risky– Autofill can generate content that appears correct but is inaccurate, misleading, or entirely fabricated. Users may trust it without realizing it’s wrong, and we know this, but those silent hallucinations or incoherent responses become dangerous when handling multiple batches simultaneously.

As shown in the image below, my prompt begins with “pays” (meaning “country” in French) and then continues in Flemish. While some values are returned correctly, others are clearly wrong. Right now, I only have five files, so the errors are obvious—but what happens if my library contains dozens or hundreds of files, or if I need to apply this across multiple document libraries? My prompt on the other hand, in Flemish, doesn’t mean anything btw 😊

7# Lack of Automation.

Automation is a big, missed opportunity. Currently, Autofill processes every row or document in the same way, without considering other columns or conditions. One major improvement—thankfully—is that you can perform a “backfill” to autofill existing files; however, a “future fill 😊” feature does not yet exist.

For example, if you want to update only the items where Column X = “Approved,” or skip rows where the status is “Draft,” there’s no built-in way to do it—you must filter and handle it manually. Or imagine a library where some documents are confidential and shouldn’t be processed: today, Autofill doesn’t check for that, so you risk generating data where you shouldn’t. Adding conditional logic like this would turn Autofill from a semi-automatic helper into a brilliant, time-saving tool, reducing repetitive work and avoiding mistakes.

8# Compliance

Autofill columns (list columns) would be significantly more powerful if they could somehow be elevated to an autofill Site Column. This could then allow future Autofill columns to automatically suggest an existing Autofill Site Column if it matches the data type of the new Autofill column (and if the data looks similar).  

When I look at autofill columns through a compliance lens (which many customers will start to do once they want to use some of the Purview automation capabilities against content in these libraries), it becomes essential for (internal) column names to be consistent and reliable, especially if you need Purview controls to apply based on some column value automatically. Although Purview can certainly work against list columns, it provides the best scalability and manageability if it can be configured to work across many/all site(s), and for that, it needs consistency in metadata. 

• Example: auto-apply a retention label based on a piece of (consistent) metadata 
• Example: applying a DLP control based on a piece of (consistent) metadata

Autofill Column Names generated 

Autofill columns should generate internal names without spaces while preserving spaces in the display names. Failing to do so creates unnecessary downstream issues (such as search which also affects Purview), especially in automated column creation processes where this is easily avoidable. 

Sidebar (not Autofill) 

The terminology used for the new ‘Classification’ is confusing. The word ‘classification’ has a very different meaning for compliance people and will no doubt cause confusion. It would be better named “Document Category” or “Document Type” or “Content Type”… in fact, it would be wonderful to tie this back to an actual content type. 

#9 Confidence score with workflow

With Autofill, all extracted values are treated equally, even when some are uncertain. The user does not have a clear understanding of the accuracy of the metadata extracted from the documents. This should be a simple calculation of the confidence score based on all the column values extracted from the document. If we apply Autofill to an extensive document library, users cannot validate the values. A confidence score within a workflow can give users peace of mind that the documents are correctly tagged. For example, if the confidence score for all the fields for a specific document is less than 80%, then automatically send an email to the user notifying them about the results, so they can review the document and correct the right columns.

#10 Cross-Document Learning (Tenant-Level Model Intelligence):

Autofill models are isolated per document library — each learns independently, even when documents across libraries are similar. Introduce tenant-level model intelligence that aggregates learning across all Autofill models (with admin control). For example, if 20 libraries contain “Invoice Number,” the global model should improve in detecting that pattern everywhere. Organizations reuse document types across sites (invoices, contracts, proposals). A global learning model reduces setup time and improves accuracy exponentially.

11# The new library creation process.

A highly valuable enhancement would be to integrate Autofill columns into the new library creation process. When a user creates a new library, a guided setup or wizard could walk them through the initial steps, such as uploading documents. During this process, the system could intelligently suggest Autofill columns based on the uploaded content. This would function similarly to the Knowledge Agent, but the experience could be automatically triggered, making the setup faster and more intuitive.

Conclusion.

And that’s a wrap—those were some key points about Autofill in SharePoint’s Knowledge Agent that I believe everyone should know. I hope you found them helpful, or at least a little eye-opening, especially if you’ve been swept up in the endless hype online. While Autofill brings some impressive capabilities, it’s clear there’s still room for improvement—such as conditional formatting, more innovative automation that respects other columns and rules, better auditability and undo options, stronger language and translation support, and safeguards against hallucinations. That was the feedback I received from my customers at the time, and things may have changed since then. Plus, I might be the only one asking for these improvements, but still…

For those of us working with multiple document libraries, complex workflows, or multilingual content, these limitations are not minor—they’re real productivity blockers. If addressed, Autofill could go from being a helpful assistant to a truly brilliant, time-saving tool across your entire SharePoint environment.

Microsoft, if you’re reading this: please take these points to heart. We, the MVPs and everyday users, see both the potential and the gaps. Fixing these would make Autofill not just “good,” but game changing. Let’s make it smarter, safer, and more flexible—so we can all stop working around it and start working with it. Thanks for listening, and I can’t wait to see what comes next! 😊

Hope that helps,