About the DR Homes crawler

Who we are, how we crawl, and how to opt in or opt out.

Attribution first Sitemap only Polite rate robots.txt respected

Who we are

DR Homes (homes.islatodo.com) is a Dominican Republic real-estate aggregator built by IslaTodo — a locally-owned pocket guide to the DR run out of Sosúa. We link out to every source we index, and we always show the original agent.

User-Agent

Our crawler identifies itself as:

Mozilla/5.0 (compatible; IslaTodo-MLS-Aggregator/1.0; +https://homes.islatodo.com/about-crawler)

How we crawl

Sitemap-only discovery. We read sitemap.xml files that publishers already expose for search engines. No brute-force URL enumeration, no ID scanning.
Rate limit: at most 1 request per second per portal.
robots.txt respected. If your robots.txt disallows the * user-agent from your sitemap paths, we skip that portal automatically.
Once-nightly. Our full crawl runs at 3 AM Dominican Republic time (7 AM UTC), off-peak.
Deep-links only. We parse structured data (schema.org RealEstateListing, OpenGraph) that you already expose for Google — the same doorway.

What we store — and don't

Facts: title, price, beds, baths, coordinates, property type, town.
Photos: URL references only (hot-linked from your CDN, never copied to our servers).
Snippets: at most 200 characters of description, cited as an excerpt with a link back to your original listing (fair use).
Attribution: every rendered listing shows your source name + a prominent "View original at [your portal] →" button that sends the visitor back to your page.

Opting out

If you'd rather we not index your portal, either:

Add a Disallow: / under User-agent: IslaTodo-MLS-Aggregator in your robots.txt — we'll skip you on the next run.
Email [email protected] and we'll remove your portal from our crawler config within 24 hours.

Opting IN — for agents and agencies

If you're a DR real-estate agent and you'd like your listings featured directly (without us waiting for a sitemap crawl), you can publish an XML/CSV/JSON feed and we'll poll it every 6 hours. Free. Fully attributed. Leads route back to your WhatsApp / phone. Email us at [email protected] to set this up.

Contact

Heath Wilson · Sosúa, Dominican Republic · [email protected]