Meta has backed down. The new business terms that would have let it crawl the website of every business using its ad and commerce tools are now paused indefinitely, and they will not start on 20 November 2026 as planned.
The U-turn came after a month of pushback: court orders in Norway and Brazil, a formal investigation by the UK’s Competition and Markets Authority, letters from US senators and loud protests from publishers.
If you run a website and use Facebook ads, a product catalog or a Facebook or Instagram business account, this affects you. Below is the full timeline, what the terms said, what is still in force, and the exact robots.txt rules to control Meta’s crawlers on your own site today.
- What: Meta’s revised Commercial Terms would have let its bots “index, cache, analyze, collect, retrieve, or crawl” websites linked to any Meta business account, with no opt-out and no payment.
- Now: on 5 November 2026 Meta posted in its Business Help Center that four sets of updated terms are “paused” and “will not take effect on November 20, 2026”. The old terms stay in force.
- Why: injunctions in Norway (28 Oct) and Brazil (3 Nov, R$1 million a day fine), a UK CMA probe (30 Oct) and pressure from publishers and US senators.
- Not over: Meta has not withdrawn the clauses, and the CMA investigation continues into January 2027.
- Do this: add robots.txt rules for
meta-externalagent(AI training) and, if you want,meta-externalads. Keepfacebookexternalhitallowed so link previews work.
What Meta Announced
On Thursday 5 November 2026, Meta removed the preview page for its proposed terms and posted a short update in its Business Help Center.
The notice says the updates “are now paused and will not take effect on November 20, 2026”. It adds that the existing terms “remain in effect” and that Meta will share more when it has more to share.
A Meta spokesperson told Nieman Lab: “We’re pausing the changes as we continue to engage with stakeholders and take our time to get this right.”
Note: the notice does not mention crawling, publishers or AI training at all. It pauses the contract change. It says nothing about how Meta’s crawlers behave today.
The four sets of terms that are paused
- Meta Commercial Terms (the one with the crawling clause)
- Product Catalog Terms
- Self-Serve Ad Terms
- Ad Creative Generative AI Terms
What the Paused Terms Said
According to the Associated Press, the new Commercial Terms authorised automated tools such as scrapers and bots to “index, cache, analyze, collect, retrieve, or crawl the domains, web pages, or URLs” tied to business accounts.
| Issue | What the draft terms did | Why critics objected |
|---|---|---|
| Scope | Covered websites linked to any business using Meta’s products | Not just publishers: shops, agencies, creators too |
| Consent | Continued use of Meta’s tools meant acceptance | No real opt-out if you rely on Facebook or Instagram for customers |
| Payment | None for crawled content | Publishers see it as free AI training data |
| AI training | Early wording said “train”; Meta removed it around 16-19 Oct and limited use to “advertising and business products” | Legal experts said the revised text still allowed broad scraping |
| AI ad content | Advertisers responsible for AI-altered ads, including features Meta turns on itself | Shifts risk for Meta’s own AI tools onto advertisers |
Sources: AP via ABC News, Nieman Lab, PPC Land. Meta’s Andy Stone said on 16 October that the update was meant to help Meta “pull accurate information about products for sale”, not to scrape news sites.
Timeline: Six Weeks of Backlash
| Date (2026) | What happened |
|---|---|
| 15 Oct | Nieman Lab reports the new terms (due 30 Oct) could let Meta scrape news sites |
| 16-19 Oct | Meta removes the word “train” and narrows use to ads and business products |
| 21 Oct | UK politicians and German publishers ask regulators to pause the terms |
| 23 Oct | Meta moves the Commercial and Self-Serve Ad Terms to 20 Nov |
| Late Oct | Senators Adam Schiff (D) and Marsha Blackburn (R) write to Mark Zuckerberg asking him to withdraw them |
| 28 Oct | Oslo District Court temporarily blocks the terms. Meta says no changes without “a full consultation” |
| 30 Oct | UK CMA opens a formal investigation |
| 3 Nov | São Paulo court suspends the crawling clause for Brazilian media association members |
| 5 Nov | Meta pauses all four sets of terms indefinitely |
The Court Orders and the CMA Probe
Norway: Oslo District Court
- Date: 28 October 2026, temporary injunction, granted without hearing Meta.
- Who sued: 10 media groups through the Norwegian Media Businesses’ Association (MBL), including NRK, TV 2, Schibsted and Amedia. AP says 180 media companies backed the request, about 97% of Norwegian newspaper circulation.
- Key point: MBL chair Lene Eltvik Vindfeld said the court agreed that “Meta holds a dominant position”.
MBL CEO Randi S. Øgrey said the pause is not enough. Her three demands: respect publishers’ scraping reservations, make any journalism use in AI a separate paid deal, and do not punish publishers who refuse.
Brazil: São Paulo state court
- Date: 3 November 2026, 44th Civil Court of São Paulo, Judge Guilherme Madeira Dezem (case 4214874-21.2026.8.26.0100).
- Who sued: ABERT (broadcasters), ANJ (newspapers) and ANER (magazines).
- Order: suspends the crawling clause, bars Meta from crawling members’ sites or using that content to train or feed Meta AI, and keeps members’ Facebook, Instagram, WhatsApp and Threads accounts active if they refuse.
- Penalty: R$1 million per day, capped at R$100 million. It covers association members only.
UK: Competition and Markets Authority
The CMA opened its case on 30 October. It is looking at whether Meta abuses a dominant position with unfair terms for publishers and advertisers. Its case page covers existing terms too, with information-gathering running to January 2027, so the pause does not close it.
The European Commission also discussed the terms with Meta and welcomed the delay, according to AP. No US lawsuit has been reported.
Who Is Affected
| You are… | What changes for you now |
|---|---|
| A business using Meta ads, catalogs or a business account | Nothing new to accept. The terms in force since 6 March 2024 still apply |
| A news publisher | No contractual crawl right for Meta for now. Keep robots.txt rules in place |
| A Brazilian ABERT/ANJ/ANER member | Court order protects your site and accounts while the case runs |
| A regular Facebook or Instagram user | Not covered by these terms. Meta can already use public posts for AI (see our guide to turning off Meta AI) |
| An advertiser using Meta’s AI creative tools | The new AI liability section is paused too. Nearly 10 million advertisers used at least one such tool in Q3 2026 (Meta CFO Susan Li) |
Meta’s Five Crawlers Explained
The terms fight is about permission. The practical control you have is robots.txt. Meta’s own developer docs list five crawlers.
| User agent | What it does | Obeys robots.txt? | Block it? |
|---|---|---|---|
facebookexternalhit |
Builds link previews (title, image) when someone shares your URL | Mostly; may bypass for security checks | No, or your shared links look broken |
meta-externalagent |
Collects content to train AI models and index it for Meta products | Yes, robots.txt is the documented control | Yes, if you do not want AI training |
meta-externalads |
Supports ads and other business products | Yes | Optional; may affect ad features |
meta-webindexer |
Improves Meta AI search results and citations | Yes | Optional; blocking can cost you Meta AI citations |
meta-externalfetcher |
Fetches a page when a user asks Meta AI or an agent to | No, may bypass because it is user-initiated | robots.txt alone will not stop it |
Note: Meta-ExternalAgent launched in July 2024. Cloudflare data cited by PPC Land shows its share of AI crawling rose from 0.9% to 7.5% in a year.
How to Control Meta’s Crawlers on Your Site
- Open your robots.txt. It lives at
https://yoursite.com/robots.txt. On WordPress, edit it in your SEO plugin or upload a file to the site root with your host’s File Manager. - Add rules for the bots you want to stop. This example blocks AI training and the ads crawler but keeps link previews and Meta AI search:
User-agent: meta-externalagent Disallow: / User-agent: meta-externalads Disallow: / User-agent: facebookexternalhit Allow: / - Decide on Meta AI search. Add
User-agent: meta-webindexerwithDisallow: /only if you do not want Meta AI to cite your pages. - Save and wait 24 hours. Meta says crawlers may cache robots.txt for up to a day.
- Test a share. Paste a post URL into the Meta Sharing Debugger (developers.facebook.com/tools/debug) and check the preview still loads.
Tip: want rules for every AI bot at once (GPTBot, ClaudeBot, Google-Extended and more) plus the new AI training policy token? Use our complete robots.txt AI crawler guide.
Warning: robots.txt does not delete content already collected, and it does not stop meta-externalfetcher. For hard blocking you need firewall rules (for example Cloudflare’s AI bot settings), which can also break features you want.
How to Check What Meta Is Crawling
- Search your access logs. On a Linux server run
grep -i "meta-external" access.log | wc -lto count hits. Hosting panels such as Hostinger and cPanel have a raw access log download. - Check the IP is really Meta. Anyone can fake a user agent. Run
whois <ip>and confirm the origin is AS32934 (Meta’s network). - Use Cloudflare if you have it. The AI Crawl Control / Bots view shows Meta-ExternalAgent requests and lets you block or allow per bot.
- Re-check after 24-48 hours. Hits from
meta-externalagentshould fall to robots.txt fetches only.
The Bigger Pattern: AI Firms and Your Content
This is the latest clash over who gets to use the open web for AI.
- Robots.txt standard: on 31 October, Google and Bing backed a shared robots.txt rule for opting out of AI training.
- Publishers’ line: robots.txt is not enough; they want a contractual right to refuse and paid licences.
- Meta’s AI push: the company is racing to fill tools like Muse Spaces with fresh data, which explains the appetite for business sites.
What Is Still Unknown
- New date: Meta gave none. A revised version could return after “consultation”.
- Clause changes: Meta has not said whether the crawling clause will be dropped or rewritten.
- Court cases: both injunctions are preliminary. Meta had five days from service to contest the Brazilian order.
- CMA outcome: could lead to binding commitments in the UK in 2027.
My Take
This is a real win for publishers, but read the wording carefully. Meta “paused” the terms. It did not cancel them, and it did not promise anything about its crawlers.
The useful lesson for small site owners is that you should not wait for a contract to protect you. Ten minutes on robots.txt gives you a clear, documented “no” for AI training while keeping the link previews that bring Facebook traffic.
I also expect the CMA case to matter more than the pause. A ruling on “unfair trading conditions” could shape how every big platform words its AI terms.
Frequently Asked Questions
Did Meta cancel the new business terms?
No. Meta paused them indefinitely on 5 November 2026. They will not start on 20 November, and the old terms stay in force, but Meta has not withdrawn the text.
Do I need to do anything in Meta Business Suite?
No. There is nothing to accept or reject now. Watch the Meta Business Help Center for any new version.
Can Meta still crawl my website for AI?
Its Meta-ExternalAgent crawler collects public pages for AI unless your robots.txt blocks it. The pause changes the contract, not the crawler.
Will blocking Meta’s crawlers hurt my Facebook traffic?
Not if you keep facebookexternalhit allowed. That is the bot that builds link previews. Blocking meta-externalagent only affects AI training and indexing.
What did the Brazilian court order?
It suspended the crawling clause for members of ABERT, ANJ and ANER, banned Meta from using their content for Meta AI, and set a fine of R$1 million a day, capped at R$100 million.
Is the UK investigation over?
No. The CMA probe opened on 30 October 2026 and covers existing terms too. Information-gathering runs to January 2027.
Does this affect my personal Instagram posts?
No. These were business terms. Public posts by regular users can already be used for Meta AI under Meta’s separate privacy policy.
Sources
- Meta Business Help Center notice on paused terms (5 Nov 2026), via Nieman Lab and PPC Land
- Nieman Lab: “Meta pulls down controversial terms of service” (Nov 2026)
- AP via ABC News: “Meta delays new business terms after outcry” (2 Nov 2026)
- Ars Technica: “Meta abandons sketchy commercial terms update” (Nov 2026)
- PPC Land: “Meta halts its Nov 20 terms” and “Meta faces R$1m daily fine if it scrapes Brazilian media sites for AI”
- Meta for Developers: “Meta Web Crawlers” documentation
- PPC Land: Meta-ExternalAgent glossary (Cloudflare data, AS32934 verification)
