The content-gap analysis that pays for itself in a coffee's worth of API credit

The whole play in one sentence

Find the pages your competitors rank for that you don’t even have in your portfolio — then turn that list into three wins: more SEO visibility, search campaigns aimed at landing pages you were missing, and non-commercial traffic you’d otherwise never touch. One map, three different payoffs, landing on three different teams. That’s the entire analysis; everything below is how you actually get there.

Every SEO deck has the slide that tells you to do this — analyze the competition, find the gaps in your content and categories. Almost nobody follows it, because the old way was a grind: export a competitor’s keywords, eyeball which landing pages they led to, line them up against your sitemap by hand, and argue about matches in a spreadsheet for a week. So the slide stayed a slide.

Two things changed. Pulling the data stopped being expensive: DataForSEO does for cents what Semrush does for a monthly subscription. And the matching stopped being manual: an LLM pairs a competitor’s pages against yours in minutes, and it doesn’t need to be perfect to be useful.

What’s left is the part that was always the real value — and it was never really an SEO task. A content gap is a missing category page, a product line you don’t stock, a blog that feeds a remarketing list. It touches SEO, paid, assortment and content strategy at once. Here’s how I run it.

One shop, four competitors — what this looks like in real life

Before the mechanics, the shape of it. Say ABC is a mid-size outdoor-gear e-shop: tents, trail shoes, rain jackets, backpacks. You pull its real competitor set and four names come back — and the interesting part is why each one beats ABC organically:

• Competitor 1 runs a genuinely good blog on how to choose a trail-running shoe. It pulls thousands of buyers at the research stage — before they’re even searching for a product.
• Competitor 2 owns the how-to content on re-waterproofing a rain jacket — a topic ABC has never written a single line about, yet every jacket owner eventually searches for it.
• Competitor 3 has a deep category tree: separate landing pages for kids’ rain jackets, 3-season tents, ultralight backpacks — granular pages that ABC currently folds into one generic listing.
• Competitor 4 simply outspends everyone on brand terms. That’s noise, not a content gap — drop it.

That one map hands ABC three concrete moves:

1. Content — write the trail-shoe and jacket-care guides, capture the research-stage traffic the blogs are eating, and drop those readers into a remarketing list.
2. Paid — once the granular landing pages exist, point search campaigns at them (kids' rain jackets → the kids’ rain-jacket page) instead of dumping every click on one blunt category page.
3. Assortment & structure — build the missing category pages for lines ABC already stocks but never gave a proper home, and flag product lines a competitor carries that ABC doesn’t.

Same analysis, three teams, three budgets. (Illustrative example.) Now here’s how you produce that map.

The flow, end to end

Watch it run: what each step actually spits out

The five steps above are the map; this is the territory. Below is the concrete artifact each step hands you — what you’re literally staring at before you move on. The shapes are exactly what the tools return; the rows are illustrative, not a real client. (Illustrative examples throughout.)

Step 1 → a competitor shortlist, scored. You run the three signals and collapse them into one table. The brands you’d have named by gut aren’t always the ones that survive all three:

Domain        Paid/Org  Overlap  Shared  Verdict
rival-a.com   yes/yes   71%      4,120   core
niche-c.com   yes/yes   44%      2,300   core
rival-b.com   no /yes   12%      3,880   content-only rival
bigbox.com    yes/yes    9%        910   too broad — drop

Three of four survive; the megastore that “obviously” competes gets dropped because the overlap is noise.

Step 2 → a page map with money attached. For each surviving competitor, one API call returns ranked keywords and the landing page each one hits. Aggregate by page and you stop looking at keywords:

Page                 Keyword             Pos  Vol     Visits/mo
/trail-shoes         trail running shoe   2   18,100  ~8,000
/waterproof-jackets  waterproof jacket    4   12,000  ~3,200
/blog/clean-shoes    clean trail shoes    1    2,400  ~1,500
/gaiters             running gaiters      6      900    ~640

Each row is a page earning a competitor real traffic — a target, not a search term.

Step 3 → your own inventory, and how much the sitemap missed. Mirror it for your site from crawl + feed + category export. The point of pulling four sources is visible the moment you count them:

Source                 Pages found
XML sitemap                  412
Screaming Frog crawl         938
Product feed               1,205 SKUs
Category export               64
Deduped own-site map       1,010 URLs

The sitemap saw 412 pages. The real map is 1,010. The analysis lives in the ~600 the sitemap never showed you.

Step 4 → the NO MATCH rows, ranked. Hand both inventories to the LLM with the matching prompt. It returns one verdict per competitor page; you keep only the gaps, sorted by traffic:

Competitor page        Closest OURS     Verdict   Visits/mo
/trail-shoes           /running-shoes   MATCH       —
/waterproof-jackets    —                NO MATCH    3,200
/blog/clean-shoes      —                NO MATCH    1,500
/gaiters               —                NO MATCH      640

One MATCH falls away; three ranked gaps remain. That four-row table is the entire deliverable in embryo.

Step 5 → the gap map with an owner. Tag each gap with a bucket and the team it lands on. Now it isn’t an SEO report — it’s a work order:

Gap                   Visits/mo  Bucket              Lands on
/waterproof-jackets   3,200      sell it, no LP      SEO / web
/blog/clean-shoes     1,500      non-commercial      Content + ads
/gaiters                640      don't stock it yet  Assortment

One map, three teams, every row sized by traffic. That’s the moment a “content-gap analysis” stops being an SEO chore and becomes a cross-team plan.

Semrush vs. DataForSEO: why the price gap matters

The reason this analysis went from “we should” to “we did” is cost — and the Semrush number that matters here is higher than the sticker price people quote. The $139.95/mo Pro plan runs a content-gap check in the interface, by hand, with export caps. But the analysis in this article is programmatic: one API call per competitor domain, ~100k ranked keywords and their landing pages at a time. Semrush gates its API behind the Business plan at $499.95/mo — and even then you start with zero API units. You buy those separately (roughly $50 per million units, ~10 units per ranked-keyword row), on top of the subscription. DataForSEO is pay-as-you-go: a $50 top-up lasts months, there’s no seat to rent or tier to unlock, and you only pay for the queries you actually pull.

For a one-off, deeply technical job like a content-gap pull, that’s the difference between unlocking a $500/mo API tier and spending a coffee’s worth of credit. The data quality is there for this use case; the economics aren’t close.

Two stories from twenty years of doing this

The mechanics are new. The plays they unlock are ones I’ve watched work for two decades — they were just too laborious to set up before.

The children’s blog that became a sales channel

A client in the kids’ segment was getting beaten on a class of queries that had nothing to do with products. The competitor ran a strong blog — coloring pages, bedtime stories — with enormous search volume aimed at exactly the target audience: parents. The gap analysis surfaced the whole cluster. The client adopted the strategy, built the content, pulled the traffic, dropped those visitors into remarketing, and turned a “non-commercial” content gap into purchases. (Anonymized.)

Recipes for a diet that sells meal boxes

A meal-prep and coaching business sat next to a category with two beautiful properties: recipe queries have extreme search volume and cent-level CPCs. The strategic competitors had built structured recipe sections — and harvested a stream of people who, by definition, wanted to eat better. From there it’s a short step to a product or a coaching offer. The gap analysis is what made the opportunity visible and sized it. (Anonymized.)

The twist nobody runs: borrow from a stronger market

Here’s the angle that turns this from a defensive audit into an unfair advantage.

Say you’re the leader in a small market with no serious competition to learn from. The gap analysis at home returns nothing useful — there’s no one ahead of you to copy. So don’t run it at home. Run the exact same analysis against the strongest, most competitive foreign market in your category.

Language isn’t a barrier: the LLM maps their categories and content onto yours regardless of the language they’re written in. You import the strategies the leaders in a mature market have already proven — category structures, content angles, assortment ideas — into a market where literally nobody is doing them yet. You become the first mover at home by copying the future from abroad. It pairs naturally with a full market-expansion analysis when you’re deciding where that stronger market is.

Why this closes the loop

Notice what just happened. We started with a tidy SEO task — “find content gaps” — and it spilled into assortment decisions, paid campaigns, remarketing audiences and content strategy. That’s not scope creep. That’s the actual shape of the work.

The data was always pullable; nobody bothered, because the manual cost outweighed the payoff. Now the pull is cheap and the matching is automated. What’s left as the scarce ingredient is the thing that was always scarce: the idea — the seniority to look at a gap map and know that a competitor’s coloring-page blog is really a remarketing channel, and the range to connect SEO, paid and assortment in one head. The execution got easy. The judgment is the job.

You are a site-structure analyst. You get two lists of pages:
COMPETITOR (url, top keywords, estimated monthly traffic) and OURS (url, title).
For each COMPETITOR page, return the single closest OURS page, or "NO MATCH".
Then output only the NO MATCH rows, sorted by estimated traffic descending.
Match on intent and topic, not exact wording. Cross-language matches are allowed.
100% precision is not required — never invent a match to avoid "NO MATCH".

curl -s "https://api.dataforseo.com/v3/dataforseo_labs/google/ranked_keywords/live" \
  -u "$LOGIN:$PASSWORD" -H "Content-Type: application/json" \
  -d '[{"target":"competitor.com","location_code":2840,"language_code":"en","limit":1000}]'

FAQ

CTA: Curious what your strongest competitor ranks for that you don’t? Let’s pull the gap map.