In This Make-or-Break Moment for Content Marketing Strategies, Should You Block AI or Embrace It?
Jen Carroll
When CloudFlare CEO Matthew Prince announced a new product designed to block ai crawlers on a recent episode of the podcast Hard Fork, he framed Pay Per Crawl as a solution to the website traffic crisis he he’d just quantified. He said it’s now 10 times harder to get discovered on Google than in 2014, 750 times harder from OpenAI, and 30,000 times harder from Anthropic.
When toggled on, CloudFlare’s Pay Per Crawl prevents bots from crawling your site by default unless AI companies pay for access. And perhaps the best part? It will be available to all CloudFlare customers, including free users. Prince says:
What’s also important is we provide this at no cost to even our free users, because we think that this is fundamentally important to the sort of health of the long-term internet. Like, it shouldn’t be something that just the big companies can get access to. And so no matter who you are, if you’re signing up for Cloudflare, you get these tools, you get the analytics, you get the understanding, you get the ability to block it.
For publishers watching their audiences disappear into AI summaries, Pay Per Crawl sounds like salvation for content monetization. For businesses that rely on content marketing strategies to generate leads, blocking AI discovery seems counterproductive.
But Prince’s traffic statistics suggest a third possibility: the entire debate about blocking AI may be missing the point.
The false promise of AI content monetization
Near the end of his interview, Prince suggested AI content licensing might evolve like digital music did—starting with iTunes’ 99-cent-per-song model before settling into something resembling Spotify’s all-you-can-eat subscription approach.
“I think we’re going to take some iterations to figure out where we might start at some fixed price,” Prince says, “and we might evolve to something which is more like a Spotify model over time.”
It’s easy to see why tech execs love this comparison. The Spotify analogy makes content licensing seem both inevitable and fair—a nice, well-worn path that supposedly benefits everyone involved. But there’s a fundamental issue with using Spotify as the model for AI content licensing: musicians are increasingly abandoning Spotify.
The Spotify model is already broken for creators
Earlier this month, NPR’s Isabella Gomez Sarmiento reported on indie bands like Deerhoof, Xiu Xiu, King Gizzard & the Lizard Wizard, and Hotline TNT pulling their music from Spotify. While their immediate trigger was CEO Daniel Ek’s AI defense investments, their deeper complaints reveal systemic problems with the streaming economy that would plague any AI content licensing system.
According to the musicians Sarmiento interviewed, they see little to no benefit from online streaming. Despite Spotify paying out $10 billion to the music industry in 2024, 69% of musicians surveyed by MusiCares cannot cover their expenses from music alone. Hotline TNT sold 300 copies on Bandcamp in 24 hours, earning more profit than months of Spotify streams. Caroline Rose’s streaming-free album became her most profitable record because she owns it entirely.
Xiu Xiu’s Jamie Stewart says, “The disposable-ness of music has become almost culturally endemic… It has not done anything good for bands.”
The problems musicians face with licensed content and streaming would likely be magnified for content creators in several key ways.
#1: The “one and done” problem gets worse
Musicians at least benefit when fans replay songs—Stewart admits Spotify represents “a noticeable amount” of Xiu Xiu’s revenue despite his complaints. A fair amount of written content has zero replay value. Once AI extracts the knowledge, there may be no ongoing relationship. Unlike music, which can be emotionally revisited, many types of content are consumed and discarded.
#2: The discovery vs. ownership tension
Musicians are increasingly choosing ownership over discovery. Caroline Rose focused on “quality rather than any and everything quantifiable” with her Bandcamp-only release and limited vinyl pressing. As Will Anderson from Hotline TNT noted: “When someone buys one of our records at a show, no one’s going to take the music off their shelf overnight like we just did with Spotify.”
Publishers face the same choice: block AI or permit discoverable exploitation. But unlike musicians, they can’t easily replicate the direct sales model that’s working for bands.
#3: The platform dependence problem
Even successful musicians feel powerless against platform decisions. As Seth Hubbard from Polyvinyl Records put it: “It’s really hard to have superhigh principles… Where do you draw the line?” Content creators would face identical dilemmas with AI licensing platforms, but with even less leverage than musicians have.
If musicians with emotional, repeatable content are abandoning the streaming model, what hope do content creators have with informational, one-time-use material? The musician exodus from Spotify reveals that even the supposedly successful streaming economy is built on creator exploitation—a model that could be even harder for written content.
The Blendle lesson: why the micropayment idea failed
It’s worth noting here that two Dutch entrepreneurs tried micropayments for written content. Blendle launched in 2014 with article-by-article payments, allowing readers to pay small amounts for individual pieces rather than full subscriptions. The Dutch startup raised millions and expanded internationally before quietly pivoting away from micropayments in 2019.
Blendle collapsed for reasons that remain relevant today: consumer friction, subscription fatigue, and discovery problems. Even small payments create decision fatigue. People prefer bundled, predictable costs over constant micro-decisions about what content is worth their money.
If individual micropayments didn’t work in the pre-AI era when people were actively seeking content, why would they work in an AI-mediated world where most users never see the original sources?
Could better execution make micropayments viable? Possibly. But the structural problems that killed Blendle would likely intensify because even perfect technical implementation can’t overcome the basic human preference for predictable costs over constant payment decisions—especially when AI already provides “good enough” answers for free.
The format-independent extraction problem
Anyone involved in audio-visual creation won’t enjoy immunity, either. Alex Reisner’s recent investigation for The Atlantic, AI Is Coming for YouTube Creators, found that at least 15.8 million videos from more than 2 million YouTube channels have been downloaded without permission to train AI products. Nearly 1 million of these are instructional videos—content that typically requires significant production investment.
Reisner highlights woodworker Jon Peters, whose YouTube channel has grown to over 1 million subscribers over 15 years. His videos require woodworking expertise, filming equipment, editing time, and often multiple takes. Yet his content—along with videos from organizations like the BBC and TED—has been used to train AI systems without compensation.
Across text, audio, video, and visual content, the core economic issue remains consistent: AI systems can extract and use content value without ongoing creator compensation, while creators bear production costs regardless of format. This explains why the strategic questions around AI blocking extend beyond traditional publishers to any business creating content.
B2C / B2B content marketing as business development
The decision to block AI or not depends on how content functions in your business. Companies that monetize content consumption directly—newsletter publishers, course creators, independent journalists—face different incentives than businesses that solve complex problems or offer distinctive products and use content to demonstrate expertise and build brand awareness.
A specialty manufacturer explaining innovative production processes, a SaaS company sharing implementation case studies, or a destination retailer showcasing unique craftsmanship—these SMBs build authority by communicating their specialized knowledge and distinctive value propositions across multiple channels.
For these companies, AI citations via generative engine optimization (GEO) represent one recognition signal among many—similar to industry referrals, trade publication mentions, or speaking opportunities. When AI systems reference your expertise, it functions as an element in comprehensive authority building that includes industry relationships, traditional SEO, media coverage, and direct business development.
The strategic question becomes how to integrate AI considerations into existing content marketing strategies rather than treating it as a separate optimization challenge—ensuring your expertise is clearly documented and easily attributable while maintaining diverse marketing approaches.
Three content monetization futures to consider
Interestingly, as the current AI scraping crisis increases, there are signs that printed marketing materials are again on the upswing. Perhaps, like me, you’ve noticed more printed catalogs in your mailbox again. In large part, I’m sure it’s because companies are trying to break through online attention fatigue, but perhaps print will also be revived as an AI blocker of sorts.
Physical content requires individual effort to digitize. Someone must photograph, scan, or manually transcribe each piece. This friction provides a certain level of protection digital content completely lacks, since entire websites and databases can be scraped automatically with zero effort. Hybrid physical-digital or print-only content marketing strategies could become viable again—not because AI can’t process physical content, but because the time and effort involved makes mass extraction impractical.
But physical publishing is just one potential adaptation. The fundamental question remains: how will content monetization evolve as AI reshapes discovery? Three scenarios capture a range of possibilities:
Scenario 1: The new information aristocracy
AI companies and major publishers might negotiate licensing deals that exclude smaller creators entirely. Knowledge creation could become stratified with well-funded sources receiving compensation while independent voices get marginalized. This resembles what Prince warned about in his Hard Fork interview—a return to the House of Medici model where knowledge creation serves power rather than public understanding. Only instead of wealthy families funding scholars, it would be tech companies, governments, or other powerful social systems determining which information sources deserve compensation.
Scenario 2: The content stratification system
Content monetization might separate into distinct tiers—ranging from mass digital content to private networks requiring industry connection. This creates a system where access to increasingly protective tiers depends on having money, connections, and technical resources rather than content quality, and the gatekeeping mechanisms would probably mirror traditional media inequalities—favoring creators with financial resources, technical knowledge, established networks, or geographic advantages. The “choice” becomes illusory when most creators lack the prerequisites for higher tiers.
Scenario 3: Business model-specific strategies
Different types of content creators develop entirely different approaches. Content entrepreneurs focus on direct relationships and premium subscription tiers. SMBs use a two-tier strategy: freely available thought leadership content designed for AI discovery combined with gated conversion content requiring direct engagement. Hybrid businesses navigate both approaches carefully, making strategic choices about when to block AI and when not to.
Implications for SMB content marketing strategies
Given the uncertain future and economics of AI and content monetization, smart SMBs must focus on strategies that remain valuable regardless of how discovery channels evolve.
For content-heavy businesses
Companies that rely significantly on content consumption face the greatest challenges, and their response requires diversifying beyond content-dependent revenue streams and building direct customer relationships. This might involve consulting services, speaking engagements, product sales, or, as I mentioned earlier, going back to printing for at least some content. Having clear value propositions and messaging becomes crucial when customers need to understand why they should engage directly rather than settling for AI summaries.
For SMBs with specialized expertise or distinctive offerings
SMBs that solve complex problems or offer distinctive products face a more nuanced calculation than just block AI or invest in SEO and GEO. Those businesses need content marketing strategies that protect their competitive advantages while building broader market recognition.
If B2C or B2B content marketing makes sense for your company:
- Focus on building brand awareness and communicating your specialized knowledge across multiple channels.
- Articulate your expertise in ways that demonstrate genuine competence and support various stages of the customer journey.
- Manage internal expectations. Building authority around your capabilities requires time, effort, and consistency.
Ultimately, I believe the goal isn’t to choose between AI-friendly or AI-resistant strategies, but to build recognition systems that deliver value regardless of how potential customers discover you.
No easy answers, only strategic choices
I’m not yet convinced anyone, including CloudFlare, will be able to come up with a solution to AI scraping that benefits all businesses equally. It’s far too simplistic a hope for our times. Different business models require fundamentally different strategies, and what helps one type of company may harm another. The promise of democratized internet publishing—where anyone could reach global audiences—may be evolving into something more stratified and complex.
This isn’t necessarily catastrophic, but it does require strategic thinking beyond a false dichotomy: block AI altogether or continue on the same marketing path many SMBs have been on for the past two decades—create content, optimize it, and hope for the best.
Rather than waiting for a clear direction that may never emerge, SMBs can choose strategies now based on their actual business models and customer relationships. The companies that succeed will be those that create and implement marketing plans designed to outlast: developing expertise worth discovering, communicating it clearly across multiple channels, and building relationships that provide value regardless of how people find you.
Whether AI systems help or hinder discovery becomes less critical when your business is built on genuine competence and authentic customer connections.
If you’re struggling to navigate the AI content dilemma or need help developing content marketing strategies that work regardless of how customers discover you, contact us. We specialize in helping SMBs build genuine authority and expertise recognition that outlasts algorithm changes and platform shifts.
TL;DR
CloudFlare’s Pay Per Crawl promises to solve the AI scraping crisis by blocking crawlers unless they pay content creators. But the proposed solution—a Spotify-like licensing model—is already failing the creators it claims to support. Musicians are abandoning streaming platforms because 69% can’t cover expenses despite billions in payouts.
Content creators face even worse economics than musicians: written content lacks replay value, creators can’t easily replicate direct sales models, and AI extraction is format-independent across text, video, and audio. Previous attempts at micropayments (like Blendle) failed due to consumer friction and decision fatigue—problems that would intensify when competing with free AI summaries.
The real strategic choice isn’t whether to block AI, but understanding how content functions in your business. Companies that monetize content consumption directly face different incentives than those using content to demonstrate expertise and build authority.
Three potential futures emerge: an information aristocracy where only well-funded sources get compensated; a stratified tier system based on resources rather than quality; or business model-specific approaches where different creators adopt entirely different strategies.
SMBs should focus on strategies that remain valuable regardless of discovery channels: building genuine expertise, communicating it clearly across multiple platforms, and developing authentic customer relationships. Physical publishing may even make a comeback as an AI-resistant marketing channel.
The future belongs to businesses that solve real problems and can communicate that value clearly, whether customers discover them through traditional search, AI references, or direct relationships.