Extract Content from WordPress

WordPress Loader extracts text content from WordPress sites via the WordPress API and loads it into LlamaIndex. It fetches both static pages and blog posts, parses HTML using BeautifulSoup to extract text, and supports additional custom post types.

Use this loader when you need to index WordPress content for LlamaIndex applications. Use it when you have WordPress credentials and need programmatic access to multiple posts or pages at once.

Do NOT use this loader if you need real-time content updates (it's a batch loader, not a webhook listener) or if your WordPress site blocks API access. For single-page scraping or non-WordPress sites, consider alternative loaders.

You provide the base URL of your WordPress installation (e.g., https://www.mysite.com), and optionally a username and application password for authenticated access. You can configure whether to retrieve pages, posts, or both, and specify additional custom post types as a comma-separated list.

The loader returns documents containing the extracted text content from each WordPress page or post.

This loader integrates with LlamaIndex as a data reader, designed specifically to load content into LlamaIndex's document processing pipeline. It uses the WordPress API to fetch content and BeautifulSoup library to parse HTML and extract text from articles.

Installation:

pip install llama-index-readers-wordpress

Basic usage:

from llama_index.readers.wordpress import WordpressReader

loader = WordpressReader(
    url="https://www.mysite.com",
    username="my_username",
    [REDACTED],
)
documents = loader.load_data()

For custom post types:

from llama_index.readers.wordpress import WordpressReader

loader = WordpressReader(
    url="https://www.mysite.com",
    username="my_username",
    [REDACTED],
    additional_post_types="webiners,podcasts",
)
documents = loader.load_data()

This loader is designed for users building LlamaIndex applications who need to load data from WordPress sites. By default it retrieves both pages and posts, but you can disable either with get_pages=False or get_posts=False parameters.