Scrape and Query Brazilian Notary Public Auctioneer Data

Junta Leiloeiros collects and serves public data on official auctioneers (leiloeiros) registered with all 27 Brazilian state Commercial Boards (Juntas Comerciais). A multi-state scraper gathers the registries, stores them in a local SQLite database, and exposes them through a REST API and CSV or JSON exports. Only public pages of the boards are accessed, with no internal systems, restricted areas or private APIs.

The scraper package has an abstract base class, a registry of the 27 state scrapers, dedicated scrapers for states whose pages need special handling (for example JUCESP for SP, JUCEMG for MG and JUCEC for CE) and a generic scraper with automatic table and list detection, used for states such as ES, SE, GO, MT, MS, AM and RR. Sites that render with JavaScript (RJ, RS and PE) use Playwright with headless Chromium; the rest use httpx with BeautifulSoup. run_all.py orchestrates collection, db.py prints per-state statistics, serve_api.py runs the API, and export.py writes files.

Collected records follow a single leiloeiros table: state, board name, registration number, full name, CPF or CNPJ when available, status, address, city, phone, email, registration and update dates, source URL and a UTC scrape timestamp, with a uniqueness rule on state plus registration number that replaces older rows. Status text from each board is normalized: Ativo, Regular, Habilitado or Regularizado become ATIVO; Cancelado, Baixado or Extinto become CANCELADO; Suspenso and Irregular keep their own values; anything else is kept as received.

The scraping is polite by design: 2 seconds between requests per domain, at most 3 retries with exponential backoff, at most 5 scrapers running at once by default, and no deletion of history.

The reference table for the 27 boards records each board's URL, collection method and status: CUSTOMIZADO for a dedicated scraper, GENERICO for the generic detector, PLAYWRIGHT for pages that need browser rendering and INDISPONIVEL for a site that is down or has no auctioneer page, with the failure recorded in the log. After each collection, boards with status VAZIO or ERRO in the log are re-checked for changed URLs or a need for browser rendering. Registry access is described as public by nature, with matriculation, name and registration status as public data and contact data handled with care.

Use it when the task mentions auctioneers at a state commercial board, such as "leiloeiro junta", "scraper junta", or a specific board like JUCESP, JUCERJA or JUCEMG: building or refreshing a list of official auctioneers, checking an auctioneer's registration and status, or exporting the data.

Do not use it for unrelated work or where a simpler, more specific tool suffices. It is not a substitute for environment-specific validation, testing or expert review. On the legal side, the registry data is treated as public professional registration data, citing Decree 21.981/1932, IN DREI 72/2019, LGPD articles 7 (II and III) and 13, and the Access to Information Law 12.527/2011. Personal contact data such as CPF, email and residential phone must be handled carefully and used only for professional identification, nothing should be published or passed to third parties without additional consent, and large-scale commercial use should involve formal contact with the boards.

Setup installs the requirements and, for JavaScript sites, the Playwright Chromium browser. A collection run accepts state codes, a dry-run mode that shows what would be collected without running, and a concurrency setting (default 5):

python C:\Users\renat\skills\junta-leiloeiros\scripts\run_all.py --estado SP RJ MG

Outputs are the SQLite file data/leiloeiros.db, a per-collection log in data/scraping_log.json and exports in data/exports/. Export supports --format csv, json or all, optionally filtered with --estado SP. The FastAPI server offers interactive docs at localhost port 8000 and these endpoints: GET /leiloeiros (filters estado, situacao, nome and limit), GET /leiloeiros/{estado}, GET /busca?q=texto, GET /stats, GET /export/json and GET /export/csv. From Python you import Database from db.py, call init(), then get_all(estado="SP", situacao="ATIVO"), search("silva") or get_stats().

To support a state that needs custom logic, subclass AbstractJuntaScraper, set the state, board name and URL, implement parse_leiloeiros, and register the class in scripts/scraper/states.py. After each run, check the log: a state with status VAZIO likely has a changed URL, and one with ERRO may need Playwright.

The stack is Python with httpx, BeautifulSoup, Playwright, SQLite and FastAPI. URLs for all 27 boards were verified on 2026-02-25, and 13 confirmed migrations of old domains are recorded (for example jucers.rs.gov.br to jucisrs.rs.gov.br). For the AP board, insecure TLS is allowed only when JUNTA_INSECURE_TLS=1 is set. When a board site is down, fallback sources are DREI, BomValor, InnLei and FENAJU, and web_scraper_fallback.py --estado MA RN AP calls the web-scraper skill for extra extraction. Companion skills are leiloeiro-avaliacao, leiloeiro-edital, leiloeiro-ia, leiloeiro-juridico and leiloeiro-mercado.

It suits developers, legal and market researchers, and auction-industry teams in Brazil who need a current, queryable list of official auctioneers across all states, with the data delivered as a database, an API or exports.