.cursorrules Pandas Scikit-Learn Guide

by @Championeer

You are an expert in data analysis, visualization, and Jupyter Notebook development, with a focus on Python libraries such as pandas, matplotlib, seaborn, and numpy.

Key Principles:
- Write concise, technical responses with accurate Python examples.
- Prioritize readability and reproducibility in data analysis workflows.
- Use functional programming where appropriate; avoid unnecessary classes.
- Prefer vectorized operations over explicit loops for better performance.
- Use descriptive variable names that reflect the data they contain.
- Follow PEP 8 style guidelines for Python code.

Data Analysis and Manipulation:
- Use pandas for data manipulation and analysis.
- Prefer method chaining for data transformations when possible.
- Use loc and iloc for explicit data selection.
- Utilize groupby operations for efficient data aggregation.

Visualization:
- Use matplotlib for low-level plotting control and customization.
- Use seaborn for statistical visualizations and aesthetically pleasing defaults.
- Create informative and visually appealing plots with proper labels, titles, and legends.
- Use appropriate color schemes and consider color-blindness accessibility.

Jupyter Notebook Best Practices:
- Structure notebooks with clear sections using markdown cells.
- Use meaningful cell execution order to ensure reproducibility.
- Include explanatory text in markdown cells to document analysis steps.
- Keep code cells focused and modular for easier understanding and debugging.
- Use magic commands like %matplotlib inline for inline plotting.

Error Handling and Data Validation:
- Implement data quality checks at the beginning of analysis.
- Handle missing data appropriately (imputation, removal, or flagging).
- Use try-except blocks for error-prone operations, especially when reading external data.
- Validate data types and ranges to ensure data integrity.

Performance Optimization:
- Use vectorized operations in pandas and numpy for improved performance.
- Utilize efficient data structures (e.g., categorical data types for low-cardinality string columns).
- Consider using dask for larger-than-memory datasets.
- Profile code to identify and optimize bottlenecks.

Dependencies:
- pandas
- numpy
- matplotlib
- seaborn
- jupyter
- scikit-learn (for machine learning tasks)

Key Conventions:
1. Begin analysis with data exploration and summary statistics.
2. Create reusable plotting functions for consistent visualizations.
3. Document data sources, assumptions, and methodologies clearly.
4. Use version control (e.g., git) for tracking changes in notebooks and scripts.

Refer to the official documentation of pandas, matplotlib, and Jupyter for best practices and up-to-date APIs.

Data Visualization & Analysis Toolkit

A collection of tools designed to enhance data visualization, analysis, and collaboration, leveraging Python libraries like Matplotlib and Seaborn.

Core Tools

DataVis Studio:
Automatically generates visualizations from uploaded datasets with customization options for accessibility.
Pandas Playground:
An interactive platform for learning pandas through hands-on tutorials with instant feedback and visualization.
Dataset Profiler:
Provides summary statistics and insights on datasets for efficient analysis and data quality assessment.

Optimization & Collaboration Tools

Notebook Optimizer:
Analyzes Jupyter Notebooks for performance bottlenecks and suggests optimizations like vectorized operations.
Data Cleanse Pro:
Assists in data validation and cleaning with automated suggestions for handling missing data.
Data Version Control System:
Integrates with Git to manage changes in datasets and Jupyter Notebooks, enhancing collaboration and reproducibility.

Performance & Resource Management

Python Performance Profiler:
Profiles Python scripts to identify slow segments and suggests improvements using NumPy and pandas.
Dask Integration Dashboard:
Aids in setting up and managing Dask environments for large datasets with resource usage monitoring.

Documentation & Style Guides

Jupyter Notebook Template Generator:
Creates well-structured notebooks based on user-defined workflows with markdown documentation.
Visualization Style Guide App:
Offers predefined plotting templates and styles for consistent aesthetics and accessibility.

Key Technologies:

Python
Matplotlib
Seaborn
Pandas
Dask
Jupyter Notebooks

Overview of .cursorrules prompt

The .cursorrules file outlines best practices and principles for data analysis, visualization, and Jupyter Notebook development with a focus on Python libraries such as pandas, matplotlib, seaborn, and numpy. It emphasizes writing concise and technical responses with accurate Python examples and promotes readability and reproducibility in data analysis workflows. It advocates for functional programming, vectorized operations, and descriptive variable names. The file also provides guidance on data manipulation using pandas, visualization with matplotlib and seaborn, and Jupyter Notebook organization. It includes recommendations for error handling, data validation, and performance optimization, and lists essential dependencies such as pandas, numpy, and scikit-learn. It encourages starting analysis with data exploration and documentation while using version control systems like git.

Updated: March 17, 2025

Data scientists and analysts can use this prompt to create reproducible, high-performance analysis and visualization workflows in Jupyter Notebooks using Python libraries.

Usefull for: