Ingest Custom Logs into Azure Monitor

The Azure Monitor Ingestion SDK for Python enables programmatic upload of custom logs to Azure Monitor Log Analytics workspaces through the Logs Ingestion API. It handles authentication via Azure Identity, automatically splits large log sets into 1MB chunks, compresses each chunk with gzip, and uploads them in parallel without requiring manual batching logic.

Use this skill when you need to send application logs, telemetry data, or custom metrics from Python applications to Azure Monitor Log Analytics. It fits scenarios where you have structured log data that must match a predefined Data Collection Rule (DCR) schema and target custom tables ending with _CL. Use the async client for high-throughput ingestion scenarios.

Do NOT use this skill if you lack the required Azure infrastructure (Log Analytics workspace, Data Collection Endpoint, Data Collection Rule, and custom table). Do NOT use it as a substitute for environment-specific validation or testing of your DCR schema definitions.

You provide three environment variables: AZURE_DCE_ENDPOINT (the ingestion URL), AZURE_DCR_RULE_ID (the DCR immutable ID), and AZURE_DCR_STREAM_NAME (the stream name from your DCR). You also supply log data as a list of dictionaries, where each dictionary represents a log entry with fields matching your DCR schema. The TimeGenerated field is required for all logs.

The SDK invokes an optional on_error callback for partial failures, allowing you to access failed log entries through the error object for retry logic.

Azure Monitor Log Analytics - Target workspace for custom logs and queries.

Azure Identity - Provides DefaultAzureCredential for authentication across local development, managed identities, and service principals.

Data Collection Endpoint (DCE) - The ingestion endpoint URL for your Azure region.

Data Collection Rule (DCR) - Defines schema, transformations, and destination table mappings.

Sovereign Clouds - Supports Azure Government and other sovereign clouds via AzureAuthorityHosts and custom credential scopes.

pip install azure-monitor-ingestion
pip install azure-identity

Configure environment variables:

### Data Collection Endpoint (DCE)
AZURE_DCE_ENDPOINT=https://<dce-name>.<region>.ingest.monitor.azure.com

### Data Collection Rule (DCR) immutable ID
AZURE_DCR_RULE_ID=dcr-xxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxx

### Stream name from DCR
AZURE_DCR_STREAM_NAME=Custom-MyTable_CL

Initialize the client:

from azure.monitor.ingestion import LogsIngestionClient
from azure.identity import DefaultAzureCredential
import os

client = LogsIngestionClient(
    endpoint=os.environ["AZURE_DCE_ENDPOINT"],
    credential=DefaultAzureCredential()
)

Python developers and DevOps engineers who operate Azure-based infrastructure and need to centralize custom application logs in Log Analytics. Data engineers building observability pipelines that consolidate logs from multiple sources into Azure Monitor. Site reliability engineers implementing error-handling patterns with retry logic for partial upload failures.