ChatExport Need Miner: Offline Market Signal & Pain-Point Extractor

ChatExport Need Miner turns an offline Telegram Desktop chat export (result.json, single file or multi-file directory) into a quantified, quote-grounded ranking of unmet market needs and product opportunities. It reads the export in fixed 4MB chunks with a 4KB overlap tail so a 500MB-5GB file never has to be loaded into memory at once, keeping the RAM footprint under 8MB.

Every hit is matched against bilingual seed lexicons (Russian phrases like "не хватает", "вот бы", "бесит", "ищу инструмент"; English phrases like "i wish", "missing", "annoying", "looking for a tool") and clustered into named themes. Themes are ranked with a Cross-Chat Multiplicity score, Score = U * sqrt(H), where U is the number of distinct chats a signal appears in and H is the total verified keyword hits - so a complaint repeated by one user in one chat is treated as an anecdote, while the same complaint surfacing across three or more independent chats is treated as a market signal.

CHUNK_SIZE = 4 * 1024 * 1024  # 4 MB
OVERLAP = 4096                 # 4 KB

Every reported theme must be backed by 2-5 verbatim quotes carrying an exact ISO timestamp and chat identifier; paraphrased or synthesized quotes are forbidden, and an unconfirmed theme is explicitly labeled [UNCONFIRMED / NO VERBATIM EVIDENCE] rather than asserted. Service messages (system notifications, bot commands, "pinned a message", "joined group") are filtered out so they are never counted as customer pain points.

Use this skill when someone hands over an offline Telegram Desktop export and asks for market research, customer-problem analysis, or tool-opportunity discovery - questions like "what are people in this chat struggling with" or "find product ideas from this export". It is built specifically for multi-megabyte or gigabyte JSON dumps where a plain json.load() risks an out-of-memory crash, and for cross-referencing complaints across several independent communities to filter out echo-chamber noise.

Do not use it for live channel scraping, continuous bot monitoring, or MTProto API automation - those need a dedicated online fetcher. It is not built for analyzing personal one-on-one relationships, and it is not a fit for structured SaaS helpdesk feeds such as Zendesk, Intercom, or Gong, which already carry ticket schemas.

Input is a Telegram Desktop result.json export, or a directory of such files, processed via chunked streaming with regex matching against the seed lexicons. Output is a set of ranked need-cluster records, each carrying a theme name, aggregate score, the number of distinct chats and total hits, the list of chats observed, the verbatim evidence quotes with timestamps, and an assessment of existing tool coverage with the specific gap each competitor leaves open. A pre-emit quality gate checks memory protection, boundary integrity across chunk cuts, verbatim quote integrity, cross-chat breadth, and that service messages were stripped before any summary is produced.

People doing market or product research from offline Telegram community exports - validating problem hypotheses, scouting for tool or feature gaps, or prioritizing what to build next based on what multiple independent communities are actually complaining about, rather than a single loud user.