The xxxtestlab-lab robot

If you found xxxtestlab-lab in your server logs, this page is about it. xxxtestlab is a site that reviews and compares adult sites. It runs two small robots, both named xxxtestlab-lab:

• The browser robot (xxxtestlab-lab/0.2, in a phone browser's User-Agent) visits the sites we review the way a new visitor on a phone would, to count pop-ups, redirects and ad trackers and to see which age check a visitor meets.
• The text robot (xxxtestlab-lab/0.1) reads the public text of a few pages of a site, three times in about two weeks, to count how often certain words appear in the titles shown to every visitor.

What the browser robot does

• It reads robots.txt first and obeys it, for every page it opens. If your home page is disallowed for it, it requests nothing else.
• Five pages per site per visit: the home page, then up to four links of your site, opened by clicking them, at least three seconds apart. A visit happens about once a month.
• From the United States it also opens, when your home page links them, your 2257 page and your copyright or content-removal page, to see whether a 2257 statement is published and how a person gets a video removed (a form or an email address). It never sends that form.
• It also notes whether your home page fits a phone screen, whether your cookie banner lets a visitor refuse (it never clicks the banner), whether the Back button returns to the page before (it presses Back once) and, on a free video site, whether a video page asks for a log-in.
• It runs your page's scripts like any phone browser, but never downloads images, videos, audio or fonts, never takes a screenshot, never logs in, never fills in a form and never pays.
• If a visitor must say "I am 18" with a button, it clicks that button once, as any visitor does. It never goes around a real age check (an ID, a face scan, a card or a provider): it records that the check is there and leaves.
• It keeps only numbers and short labels: how many pop-ups, redirects, trackers and ad frames, which kind of age check, and a percentage for the titles. It does not keep your titles, your text or any picture.
• It runs from a server in New Jersey, United States (New York area), and one in London, United Kingdom.

What the text robot does

• It reads robots.txt first and obeys it. If your home page is disallowed for it, it requests nothing else.
• At most about eight requests per site per visit: the home page, one category page, one "newest" page and up to three legal pages (2257, DMCA, terms).
• At most one request every three seconds to your site, or slower if your robots.txt sets a crawl-delay.
• Text only: it never downloads images, videos, scripts or style sheets, never runs JavaScript, keeps no cookies, never logs in, never submits a form and never goes around an age check. If a page is behind an age check, it stops and records that it could not measure the site.
• It keeps only numbers: counts, percentages and the ids of the words it looked for. It does not keep your titles, your text or any picture.

How to stop it

Add these two lines to your robots.txt. Both robots read the file at the start of every visit:

User-agent: xxxtestlab-lab
Disallow: /

Or write to xxxtestlab@proton.me with your domain: we add it to our stop list and neither robot sends anything more to it.

Robots: xxxtestlab-lab/0.2 (browser) and xxxtestlab-lab/0.1 (text). Operator: xxxtestlab. Servers in New Jersey, United States, and London, United Kingdom.