Automatic monitoring of client website accessibility

One of your clients' websites goes down. They don't notice right away. The AI models, however, see it within hours: their crawlers can no longer fetch the pages, their answers stop citing the brand, and AI visibility silently drops. By the time you run your next quarterly audit, scores have fallen twenty points and nobody knows since when.

We've seen this scenario too often. AI Labs Audit now includes automatic monitoring of client website accessibility on two levels: validation at input, and daily background checks. Here's how it works, and why it's an essential safety net for a GEO/AEO agency.

Level 1 — Immediate validation at input

When you add or update a client's URL (quick-create form or edit page), the platform performs a real-time HTTP test before saving. The "Save" button is disabled if the site returns an error code, a timeout, or a malformed URL.

• Typo detection: exampel.com instead of example.com is caught before polluting the database.
• Dead-site detection: a client who rebuilt their site on a new domain and forgot to tell you — caught at input time.
• Auto-prefix: if you type example.com without https://, the platform adds it before testing.
• Dedicated REST endpoint: GET /api/client/check-url?url=... returns ok / http_error / timeout / unreachable / invalid_url in under 6 seconds.

Result: no client record can be saved with a broken URL anymore. Audits scheduled on that client always start from a URL that was reachable the day you added it.

Level 2 — Daily background check

A site can be fine on creation day and go down three months later. That's why we run a daily cron that re-tests every active client site every night at 04:10 (server time).

The Celery task daily_check_client_websites:

1. Selects every active client with a non-empty site_web that hasn't been checked in over 20 hours.
2. Fires HEAD requests in parallel (10 workers, 6s timeout, GET fallback on 405) to avoid saturating the server.
3. Updates three columns on the client record: site_web_status, site_web_last_check, site_web_unreachable_since.
4. Resets unreachable_since to NULL as soon as a site responds again (recovery).

If more than 50% of the sites checked in a batch go KO, the platform raises an automatic CRITICAL alert in system_alerts. That's typically a server-side network or DNS issue, not a client-side problem — the agency is notified before chasing a phantom problem.

Level 3 — Visual badge on the client page

On the client detail page, if a site has been unreachable for more than 7 days, a red "Site KO" badge appears next to the domain name, with a tooltip showing the exact status (timeout, http_error, unreachable...) and the date the site went down.

Why 7 days and not immediately? To avoid false alerts on sites with scheduled maintenance lasting a few hours, or a temporary network issue. Beyond a week, you're definitely looking at a real problem that warrants agency intervention.

Real case: why this changes the game

A partner agency had 80 active clients audited monthly. About once a quarter, a client would rebuild their site without warning: new CMS, new domain, missing redirect. Before this feature, the agency only found out at report time — up to 30 days later. During that whole period, credits were burned auditing a site returning 404s.

With the integrated monitoring, the case is detected within 24 hours. The account manager calls the client the next day, updates the record, and the next audit runs on the right URL. No wasted credits, no fake reports, no nasty surprises.

Availability and activation

The feature is active by default on every plan that manages clients (Consultant, Consultant+, Agent, Agency+). No action required: validation at input is automatic, the cron runs in the background, the badge appears as soon as a client is affected. Checks consume zero credits — it's infrastructure, not a paid AI service.

Status history is queryable via the columns added to the clients table, and exposed in the REST API (GET /api/clients/{id} returns the three fields site_web_status, site_web_last_check, site_web_unreachable_since).

Conclusion

Monitoring client website accessibility isn't a glamorous topic, but it's the foundation of honest GEO/AEO measurement. Auditing a site returning 500s gives no useful information about a brand's AI visibility — it just fills the report with noise. AI Labs Audit prefers warning you before launching the audit rather than billing you for an empty report.

You can test the immediate validation right now: create a client with a deliberately broken URL in your client list, and watch the "Save" button stay disabled until you fix it.

Article reviewed and updated: May 2026 — verified against current behaviors of ChatGPT, Claude, Gemini and Perplexity.