The Complete Guide to Link and Redirect Audits

A link audit is a systematic check of every link a site exposes: internal links between its own pages, outbound links to other domains, links inside navigation and footers, URLs listed in sitemaps, and the redirects that sit between a requested address and the page that finally loads. The goal is simple to state and surprisingly hard to do well. You want to know that every URL a visitor or a search engine can reach returns the right response, lands on the right content, and does so without an unnecessary detour. Sites drift away from that state constantly. Pages get renamed, campaigns end, products go out of stock, plugins rewrite URLs, and a redirect added as a quick fix two years ago quietly becomes the first hop in a four-step chain that nobody remembers creating.

This guide is the hub for everything we have written about link audits. It is organized around seven themes: why audits matter and when to run them, how a crawler actually discovers and tests URLs, how to interpret the status codes you get back, how to untangle redirect chains and loops, how to get full coverage on large sites, how to turn raw findings into fixes that ship, and how to build a routine around launches and migrations. Each section links to the deeper article on that topic. Read it top to bottom if you are new to the subject, or jump straight to the theme that matches the problem in front of you. Nothing here depends on a particular tool. The principles apply whether you crawl with a script, a desktop application, or a hosted service.

Why Link Audits Matter and When to Run One

Broken links hurt in three distinct ways, and it helps to keep them separate because each one argues for a different fix. The first is user experience: a visitor who clicks a link in your article and lands on an error page loses trust in the page, in the site, and sometimes in the brand. The second is discovery: search engines find most of a site's pages by following links, so a broken internal link can cut off an entire branch of content from being found, refreshed, or ranked. The third is equity: links carry a signal of importance from one page to another, and a link that points at a dead URL passes that signal into a void. None of these effects show up as a single dramatic drop. They accumulate quietly, which is exactly why a periodic audit is worth the effort.

Crawl budget is the concept that ties the discovery problem to something measurable. Every crawler has a finite appetite for a given host, shaped by how fast your server responds and how much the crawler believes your site deserves attention. Each request it spends on a 404, a soft 404, or an extra redirect hop is a request it did not spend on a page you care about. On a small site this rarely matters. On a site with tens of thousands of URLs, a few thousand wasted requests per day can mean that new or updated pages wait longer to be recrawled. Our article on how broken links waste crawl budget walks through the mechanics, including why redirect chains are often a bigger drain than plain 404s, and how to read your own logs to see the waste for yourself.

The question of when to run a full audit has two answers: triggers and cadence. Triggers are events that predictably break links in bulk, such as a migration to a new domain or CMS, a redesign that changes URL structure, a large content prune, a change to how categories or product pages are generated, or a plugin update that touches permalinks. Any of these deserves a full crawl before and after the change. Cadence is the routine you keep between triggers, and it should scale with how fast the site changes. A publication that adds dozens of articles a week with outbound links needs a much tighter loop than a brochure site that changes twice a year. The article on when to run a full link audit gives a practical framework for choosing both, including what a lightweight check between full audits should cover.

How a Crawler Discovers and Tests Every URL

A site crawl begins with one or more seed URLs, usually the homepage and the sitemap. The crawler fetches each seed, parses the HTML, and extracts every link it can find: anchors, images, scripts, stylesheets, canonical tags, and sometimes links inside structured data. Each extracted URL is normalized, checked against a list of URLs already seen, and, if it belongs to the site being audited, added to a queue to be fetched in turn. External URLs are tested for their response but not followed, since the audit is about your site, not the whole web. The process repeats until the queue is empty or a configured limit is reached. That description sounds mechanical, and it is, but each step hides decisions that change what the audit reports. Our article on how a site crawl actually works goes through each of them.

Normalization is the step most people underestimate. The same page can be reachable at a dozen addresses: with and without a trailing slash, with uppercase and lowercase paths, with tracking parameters appended, with a fragment identifier, over HTTP and HTTPS, on the bare domain and the www subdomain. A crawler that treats each variant as a distinct page will report inflated counts and duplicate issues. One that collapses variants too aggressively will miss a genuine problem, such as a case-sensitive server that returns 404 for one spelling and 200 for another. Good crawlers make normalization rules explicit and let you adjust them. Politeness is the other hidden decision: a crawler that fires requests as fast as it can will produce a wave of 5xx responses from an overloaded server and then report them as broken pages, when the crawler itself was the cause.

There are also things a crawler cannot see on its own. Links injected by JavaScript after the page loads are invisible unless the crawler renders the page, which is slower and more expensive. Pages behind a login are unreachable without credentials. Orphan pages, which have no inbound internal links at all, will never be discovered by following links no matter how long the crawl runs. This is why a serious audit combines link-following with other URL sources: the XML sitemap, server logs, analytics exports, the list of URLs that search engines report having indexed, and any internal search index. The difference between the set of URLs the crawler found and the set of URLs those other sources know about is one of the most useful outputs of an audit, and it is the subject of our article on auditing a large site without missing pages.

Reading Status Codes: 404s, Soft 404s, and Everything Between

Every URL a crawler requests comes back with an HTTP status code, and the audit report is largely a sorted list of those codes. The classes are easy to remember. A 2xx response means the server delivered content. A 3xx response means the server sent the client somewhere else. A 4xx response means the client asked for something the server could not or would not provide, with 404 (not found), 410 (gone), 401 and 403 (access restricted) being the ones that matter most in an audit. A 5xx response means the server itself failed. The mistake people make is to treat 200 as automatically fine and everything else as automatically broken. Neither is true. A 200 can hide a missing page, and a 404 can be exactly the right response for a URL that should not exist.

The soft 404 is the clearest example of a 200 that is not fine. It happens when a server returns a success code but the page content says, in effect, that nothing is here: an empty product listing, a search results page with zero hits, a template that renders the chrome of the site around a blank main area, or a redirect that sends every unknown URL to the homepage. Search engines detect these by looking at the content and treat them as missing pages, which means the URL gets the same fate as a real 404 but only after the crawler has spent a request fetching and evaluating it. Worse, soft 404s tend to be systematic: one misconfigured template can produce thousands of them. Our article on 404s versus soft 404s explains how to spot them in a crawl, usually by matching on thin word counts, repeated titles, and redirect destinations, and how to fix the underlying template so that the server sends an honest code.

The rest of the non-200 responses deserve classification by intent rather than a blanket verdict. A 5xx during a crawl may be transient, caused by load or a deploy, so the audit should retry those URLs before reporting them. A 401 or 403 on a members-only page is correct and should be excluded from the broken list, though a 403 on a page that is linked from public navigation is a real problem. A 410 is a deliberate statement that a page is gone for good and will not return, which is the right choice for content you have retired and do not intend to redirect. A 404 for a URL that nothing links to is noise; a 404 for a URL that is linked from your main navigation is urgent. The audit's job is to attach enough context to each code, such as how many pages link to it and how much traffic it used to get, that you can tell the difference at a glance.

Untangling Redirect Chains, Loops, and Legacy Rules

A redirect chain is what you get when a redirect points at a URL that itself redirects. Chains almost never start as chains. They accumulate in layers: a rule that forces HTTPS, a rule that adds or strips the www prefix, a rule that normalizes trailing slashes, a migration that moved every article to a new path, and a later migration that moved them again. Each layer was reasonable on its own, but a visitor requesting an old HTTP address with the wrong slash can end up passing through four or five hops before the page loads. Every hop costs a round trip, and on mobile connections that latency is noticeable. Crawlers also follow only a limited number of hops before giving up, and every hop in between is a request spent on a detour rather than on a page you want fetched. It is a common SEO belief that a little ranking value is also shed at each hop; search engines have said that redirects pass signals, so treat that as a belief rather than a mechanism and collapse chains for the latency and crawl cost regardless.

Reading a redirect chain map is the skill that turns a list of 3xx codes into a fix plan. A good map shows, for each starting URL, every hop in order, the status code at each hop, the final destination, and the final status. The final status matters as much as the hop count: a chain that ends at a 200 is a performance problem, while a chain that ends at a 404 is a broken link wearing a disguise, and a chain that ends where it began is a loop that will never load at all. Our guide to reading a redirect chain map walks through each pattern with examples, including mixed chains where a permanent 301 hands off to a temporary 302, which sends conflicting signals about which URL is canonical. The fix for a chain is almost always the same two steps: point the first redirect directly at the final destination, then update the source links so that no redirect is needed in the first place.

Choosing the right redirect type and knowing when to remove a redirect entirely are the two judgment calls in this area. A 301 or 308 tells clients and crawlers that the move is permanent and that the new URL should become the canonical address and inherit the old one's standing. A 302 or 307 says the move is temporary and the old URL should be kept as the one to remember. Search engines have said that both kinds pass ranking signals, so the old fear that a 302 throws equity away is overstated. What a temporary code on a permanent move does is leave the canonical URL where it was: search engines keep indexing the old address, consolidation onto the new one is delayed until the redirect has been in place long enough to be read as permanent, and the migration looks unfinished in the meantime. That delay is the real cost of a wrong 302, and it is why every redirect on a permanent move should carry a permanent code. As for removal, a redirect from an old URL that still receives inbound links from other sites, or traffic from bookmarks and old emails, should be kept but collapsed to a single hop. A redirect that nothing references anymore can be retired, though it costs little to keep. Post-migration audits are where most of this cleanup happens, and our checklist for that phase covers how to verify every rule against the original URL list.

Getting Full Coverage on Large and Complex Sites

Large sites are hard to audit for reasons that have little to do with page count and everything to do with URL space. Faceted navigation on a store can generate millions of filter combinations, each a distinct URL. Pagination, sort orders, and calendar views create sequences that never end. Multiple languages, regional subdomains, and legacy sections multiply the whole thing. A crawler pointed at a site like this can run for days, consume a huge budget, and still miss the pages that actually matter because it spent its time deep inside a parameter explosion. Coverage on a large site is therefore not about crawling everything. It is about deciding what the site's real URL set is, crawling that set completely, and confirming afterward that nothing important was left out.

The practical strategy has four parts. First, gather URLs from every source you have: the XML sitemaps, server logs, analytics, the indexed URL lists from search engines, product feeds, and internal search. Together these define what the site believes it contains. Second, set include and exclude rules so the crawler ignores infinite spaces: strip or ignore known tracking and sort parameters, cap pagination depth, and exclude paths that are known to be duplicates of canonical pages. Third, segment the site and crawl in passes, one section at a time, so that a problem in one area does not starve the others of crawl time. Fourth, compare the set of URLs the crawler found against the set you gathered in step one. URLs in the crawl but not in your sources are candidates for cleanup. URLs in your sources but not in the crawl are orphans or blocked pages, and those are often where the real problems hide.

Operational limits also shape coverage. Crawling a production site at full speed can degrade it for real visitors, so crawl rate has to be throttled, which means a full pass takes longer and needs to be scheduled rather than run on impulse. Crawling staging avoids that risk but only tells you about the code, not about the live redirect rules and CDN behavior. Rendering JavaScript to discover injected links multiplies the cost per page, so it is usually reserved for a sample or for sections known to rely on client-side navigation. Our article on auditing a large site without missing pages turns all of this into a repeatable procedure, with the counts you should record at each stage so you can prove coverage rather than assume it.

Turning Findings into Fixes That Actually Ship

A raw crawl export is a list of symptoms, not a list of tasks. If a footer template links to a retired page, the crawl will report that broken link once for every page on the site, which could be ten thousand rows describing one fix. If a category page has a typo in a link, that is one row describing one fix. Treating those two situations the same way, by row count, guarantees that the important work is buried. The first step after any audit is to group issues by cause: by the template or component that produced them, by the destination URL they share, and by the redirect rule that created them. That grouping typically collapses a huge report into a manageable number of distinct problems.

Prioritization comes next, and it should be driven by impact rather than by code. A broken link in the main navigation matters more than a broken link in a five-year-old blog post. A redirect chain on a page that receives most of the site's search traffic matters more than a chain on a page nobody visits. A soft 404 pattern across an entire product category matters more than a single 404 on an obscure URL. Combine the crawl data with traffic, inbound link counts, and template scope to rank the grouped issues. Then write the export the way a developer wants to read it: one row per distinct problem, with the source page or template, the anchor text, the target URL, the observed status and final URL, the recommended action, and the number of instances affected. Our article on exporting an audit your developer will actually use shows the exact columns and why a separate raw file should accompany the summary.

The loop is not closed until the fix is verified. After the developer ships, rerun a targeted crawl on the affected URLs and confirm that the codes changed as expected, that no new chain was introduced, and that the source links now point at the final destination. Keep the previous audit as a baseline so that the next full crawl can be read as a diff: what was fixed, what is new, what regressed. A pre-launch link checklist is a good template for this verification pass, because it already lists the checks that matter for a change going live. Over time this habit turns auditing from a periodic cleanup into a quality gate, which is where it belongs.

Building an Audit Routine Around Launches and Migrations

Launches are the moment when a link audit has the highest return, because every link on the new site was written or generated recently and none of it has been tested by real traffic. A pre-launch pass should crawl the staging environment completely, confirm that every page in the planned sitemap returns 200, verify that navigation, footer, and breadcrumb links resolve, and check canonical tags, hreflang references, and pagination links for internal consistency. If the launch replaces an existing site, the old URL list is the most important input: every old URL should be tested against the new redirect rules to confirm that it lands on the correct new page in a single hop. Our pre-launch link checklist is built to be run in the final days before go-live, with each item phrased so that a non-specialist can tick it off.

The post-migration audit is a second, different pass. Once the new site is live, the redirect rules are running on real infrastructure, the CDN is caching real responses, and search engines are recrawling with their own expectations. Within the first days you should crawl the full old URL set through the live redirects, compare hop counts and final destinations against the plan, watch server logs for 404s that the plan did not anticipate, and monitor the coverage reports that search engines provide for spikes in errors or excluded pages. A follow-up pass a few weeks later catches the slower problems: chains that appeared when someone added a fix on top of a fix, pages that were dropped from the index because a canonical pointed the wrong way, and traffic that failed to transfer to its intended destination. Our post-migration audit guide lists what to check in each window.

Between launches, the routine is a matter of cadence and ownership. Schedule full audits at an interval that matches how quickly the site changes, run lighter spot checks on high-traffic pages between them, and set up alerts for the handful of URLs where a failure would be expensive, such as the homepage, top landing pages, and checkout or signup flows. Assign a single owner for the audit results, even if the fixes are spread across several people, so that findings do not sit in a spreadsheet unread. The article on when to run a full link audit covers how to size that cadence, and the rest of this guide gives you the vocabulary to act on what the audit returns. A link audit done once is a cleanup. A link audit done on a rhythm is maintenance, and maintenance is what keeps a site from needing another cleanup.

More guides on this topic

Further reading from the DeadLinkr blog, each answering one specific question in depth.

Link audits reward the boring virtues: completeness, classification, and follow-through. Crawl from every URL source you have, not just the homepage. Read status codes for intent rather than treating everything outside 200 as a bug. Collapse redirect chains to a single hop and then remove the need for the hop by fixing the source link. Group findings by cause, rank them by impact, and hand your developer a list that fits on one screen. Verify the fix, keep the baseline, and run the next audit as a diff. Do this before every launch, after every migration, and on a calendar between them, and broken links stop being a recurring emergency and become a small, predictable chore.

If you take one thing from this guide, make it this: the audit is only as good as the decisions you make about what to crawl and how to read the results. The tool, whatever it is, does the fetching. The judgment about what a 404 means on your site, which redirects to keep, and which findings to fix first is yours. The articles linked throughout are there to sharpen that judgment one topic at a time.