MailGrab: an open-source email extractor in Python

What does MailGrab do?

MailGrab starts from a URL or a list of them, follows the links it finds, collects the email addresses on each page and saves them with the page each one came from. Its GitHub description calls it an email harvester.

Finding pages

• Crawls from one start URL or from each line of a seed file, 10 pages at a time by default, with a 10-second timeout on each request.
• Follows links to other websites too, unless --same-domain keeps it on the start URL's host.
• Can stop a set number of links from the start and add the pages listed in a site's sitemaps.
• Skips links to images, PDFs, archives, scripts, stylesheets, audio, video, fonts and XML files, and tel:, javascript: and in-page # links.

Finding addresses

• Searches the source of each page for text shaped like an email address, and reads mailto: links.
• Decodes the data-cfemail form of Cloudflare's email address obfuscation and addresses written as name [at] domain [dot] com.
• Lists the LinkedIn, X, Twitter, Facebook, Instagram and contact page links it comes across, and crawls them like any other link.

Cleaning and saving

• Lowercases addresses so each is listed once, and drops 10 known placeholder addresses, such as email@example.com, along with every address at example.com, example.org, example.net and wixpress.com.
• On request, drops addresses whose domain has no mail server (MX) record.
• Saves text, CSV and JSON files, can build on an earlier run and keeps what it found if you stop it with Ctrl+C.

How to use MailGrab responsibly

An email address that relates to an identifiable person is personal data, and collecting and using it is regulated. Use MailGrab on sites you own or have permission to crawl, to find out what they expose, and not to build mailing lists.

Which laws apply?

That depends on where you are, whose addresses they are and what you do with them. Three examples:

• India's Digital Personal Data Protection Act, 2023 counts collecting digital personal data as processing it, and allows processing only for a lawful purpose, with the person's consent or for certain legitimate uses. Its section 3 sets out what the Act does not apply to. Most of the Digital Personal Data Protection Rules, 2025, published on 13 November 2025, come into force on 13 May 2027.
• The EU's General Data Protection Regulation requires a lawful basis for collecting personal data. Anyone who collects it from somewhere other than the person, public sources included, must normally tell them within a month, and people can object to direct marketing at any time. For marketing email, Article 13 of the EU's ePrivacy Directive generally requires the recipient's prior consent, and each member state applies it through its own law.
• The US CAN-SPAM Act sets rules for every commercial email, business-to-business email included, such as honouring opt-out requests promptly. The FTC's guide to the Act lists harvesting email addresses among the practices that carry criminal penalties.

What MailGrab does by default

MailGrab checks robots.txt before each page unless you turn that off on purpose with --ignore-robots or its matching setting. It sets no rate limit of its own: --delay defaults to 0 while 10 pages load at once. Add a delay for small sites. Its default User-Agent names MailGrab and links the repository, so site owners can see what visited.

On the command line, MailGrab follows links to other websites unless you pass --same-domain; add that flag when you audit your own site. The MCP tool stays on the starting site and stops at 30 pages by default, but the agent chooses each call's options: it can raise the depth, turn same_domain off or set ignore_robots, which the tool's description asks it to leave off unless it has a clear reason. The server's instructions also tell agents to use the tool whenever they need contact emails for a company or site. Check which sites your agent asks it to crawl.

RFC 9309 says its rules are not a form of access authorisation: robots.txt not blocking a page does not by itself give you permission to crawl it. An address published on a page also does not mean the person agreed to be contacted. For site owners, the standard adds that robots.txt is not a substitute for security measures. A crawler that ignores the file can still read the addresses on your public pages. Our technical SEO checklist explains what robots.txt does for search engines.

Uses that fit

• Checking your own websites, or a client's with permission, for addresses left open to scrapers, including ones written with [at] and [dot] and the data-cfemail form of Cloudflare's obfuscation, which MailGrab decodes. It runs no JavaScript: a run that finds nothing does not prove a site exposes no addresses.
• Research on sites you have permission to crawl, keeping only the personal data you have a lawful basis to keep.

MailGrab sends no email. From the command line, its results stay in files on your machine until you delete them; through MCP, they go to the agent that asked. Treat them as personal data wherever they end up.

This section describes how MailGrab works and names some of the laws involved. It is not legal advice; ask a lawyer about your own use.

MailGrab command-line options

MailGrab's command-line options set where a crawl starts, how many pages it fetches, how fast and how far it goes, whether it checks robots.txt and what it saves. None is required: with neither --url nor --depth, MailGrab asks for the start URL and the depth, or only for the depth when the seed file lists URLs. With either, it never asks: --url without --depth stops with an error, and so does --depth without --url unless the seed file lists URLs. The wiki's command-line reference explains each option in more detail.

Most options can also be set with an environment variable named after them, such as MAILGRAB_SAME_DOMAIN for --same-domain; --concurrency uses MAILGRAB_MAX_WORKERS, and --url, --depth, --input and --config have none. A switch is on only when its variable is exactly 1, and a flag beats its variable, which beats the config file. A flag can only turn a switch on: a switch set to 1 in the environment or a config file stays on until you remove it there.

A config file is a flat JSON object keyed by those variable names in lower case without MAILGRAB_, so --concurrency is max_workers there. Write a switch as "1", because a JSON true counts as off. Pass the file with --config myconfig.json; a missing or broken file stops MailGrab before it crawls anything.

{
  "delay": "0.5",
  "same_domain": "1",
  "user_agent": "MyCompanyBot/1.0"
}

Using MailGrab as an MCP server for AI agents

The repository's MCP server, mailgrab_mcp_server.py, gives AI coding agents one tool, crawl_website, which runs a MailGrab crawl and returns the results as structured data. It talks to its client over standard input and output (stdio), so the client starts it as a local process.

Each call runs MailGrab.py with --quiet as a separate process in a new temporary folder, reads _results.json back and deletes the folder. The repository gives two reasons: stdio reserves standard output for protocol messages, which the crawler's console output would corrupt, and two calls sharing a folder would overwrite each other's files. Every call also runs the startup check, so the server needs internet access.

crawl_website(
  url: str,                    # required
  depth: int = 30,
  same_domain: bool = True,
  max_hops: int | None = None,
  use_sitemap: bool = False,
  ignore_robots: bool = False,
  verify_mx: bool = False,
  delay: float = 0.0,
  timeout: float = 10.0,
  concurrency: int = 10,
  user_agent: str | None = None,
  proxy: str | None = None,
) -> {
  emails: [str],
  email_count: int,
  scrapped_urls: [str],
  url_count: int,
  sources: {email: [url]},
  social_links: [str],
}

Each parameter maps to the matching command-line option, and two defaults differ for agents: same_domain is on, unlike on the command line, and depth is 30. Finding no addresses is a normal result, and so is a site that cannot be reached: the tool returns no addresses, but scrapped_urls still lists the start URL, so an empty email list does not show whether the site answered. A malformed proxy, a depth outside 1 to 500, a failed save, a run that leaves no results file (as when the startup check fails) or a crawl that runs out of time returns a tool error with the reason. The time limit depends on depth, concurrency and timeout: 75 seconds at the defaults and 1,800 seconds at most. The tool leaves out --append, --resume and --config, because each call stands alone.

Connecting a client

To start the server by hand, run this in the repository folder:

python mailgrab_mcp_server.py

An MCP client runs the same command itself: point it at the absolute path of mailgrab_mcp_server.py, with a python command that has the requirements installed. For Claude Code, a .mcp.json file at the project root looks like this, and Claude Desktop and Cursor use the same shape in their own files:

{
  "mcpServers": {
    "mailgrab": {
      "command": "python",
      "args": ["/absolute/path/to/mailgrab_mcp_server.py"]
    }
  }
}

GitHub Copilot in VS Code uses a "servers" key instead, GitHub Copilot CLI has its own file with a tools list, and Codex CLI reads TOML. The wiki's MCP server setup page covers all six clients; the release notes say its steps were checked against each client's documentation. Once it is connected, ask your agent which email addresses your own site publishes.

How to report a bug or contribute to MailGrab

Report bugs on the MailGrab issues page on GitHub and send changes as pull requests. TODO.md lists the open items and asks you to claim one with an issue first. Under the MIT licence you can also adapt the code in your own copy.