---
name: modernize-scientific-stack
description: Guide for modernizing legacy Python 2 scientific computing code to Python 3 with modern libraries. This skill should be used when migrating scientific scripts involving data processing, numerical computation, or analysis from Python 2 to Python 3, or when updating deprecated scientific computing patterns to modern equivalents (pandas, numpy, pathlib).
---

# Modernize Scientific Computing Stack

This skill provides guidance for migrating legacy Python 2 scientific computing code to modern Python 3 with contemporary libraries and best practices.

## When to Use This Skill

Apply this skill when:
- Migrating Python 2 scientific scripts to Python 3
- Updating legacy data processing code using outdated patterns
- Modernizing scripts that use deprecated file handling, string encoding, or numerical libraries
- Converting scripts from csv module to pandas for data analysis
- Replacing os.path with pathlib for path manipulation

## Approach

### Phase 1: Complete Code Discovery

Before making any changes, ensure complete understanding of the existing codebase:

1. **Read all source files completely** - If a file read is truncated, request the full content before proceeding. Never assume file contents based on partial reads.

2. **Identify all dependencies** - Check for:
   - Import statements (standard library and third-party)
   - Configuration files (JSON, YAML, INI)
   - Data files (CSV, Excel, pickle)
   - Environment requirements

3. **Map the data flow** - Understand:
   - Input file formats and encodings
   - Data transformations applied
   - Output format requirements
   - Any intermediate files or caches

### Phase 2: Identify Migration Requirements

Common Python 2 to Python 3 migration patterns in scientific code:

| Legacy Pattern | Modern Replacement |
|---------------|-------------------|
| `print "text"` | `print("text")` |
| `unicode()` / `str()` | `str()` with explicit encoding |
| `open(file)` | `open(file, encoding='utf-8')` |
| `os.path.join()` | `pathlib.Path()` |
| `csv` module | `pandas.read_csv()` |
| `for key in dict.keys()` | `for key in dict` |
| `dict.has_key(x)` | `x in dict` |
| Manual file iteration | Context managers (`with` statements) |
| `xrange()` | `range()` |
| Integer division `/` | Explicit `//` or float division |

### Phase 3: Implementation Strategy

1. **Create the modernized script** with these priorities:
   - UTF-8 encoding for all file operations
   - pathlib.Path for all file path manipulations
   - pandas for CSV/data processing
   - Type hints where beneficial
   - Context managers for resource handling

2. **Handle configuration files** - Check for file existence before reading:
   ```python
   config_path = Path("config.json")
   if config_path.exists():
       config = json.loads(config_path.read_text(encoding='utf-8'))
   ```

3. **Create requirements.txt** - Include all dependencies with version constraints

### Phase 4: Verification Protocol

**Critical: Always verify file operations**

After writing any file, read it back to confirm:
- The complete content was written (not truncated)
- The syntax is valid
- All imports are present

**Testing sequence:**

1. **Syntax validation** - Run Python syntax check:
   ```bash
   python -m py_compile script.py
   ```

2. **Import verification** - Test all imports resolve:
   ```bash
   python -c "from script import *"
   ```

3. **Functional test** - Run the script and compare output to expected results

4. **Output validation** - Verify output format matches requirements exactly

### Common Pitfalls to Avoid

1. **Truncated file content** - Never proceed with partial file reads. If a response shows `... [truncated]` or incomplete content, request the full file before continuing.

2. **Unverified writes** - After using a write operation, always read the file back to confirm the complete content was written correctly.

3. **Encoding issues** - Always specify `encoding='utf-8'` explicitly in file operations. Legacy scripts often have implicit ASCII assumptions.

4. **Path string concatenation** - Replace all `os.path.join()` and string concatenation for paths with `pathlib.Path` operations.

5. **Missing edge case handling**:
   - Empty data files or datasets
   - Missing required files
   - Invalid data types in CSV columns
   - Stations/entities with no matching data

6. **Environment setup repetition** - When setting up environments (venv, PATH), verify the setup persists rather than repeating in each command.

## Verification Checklist

Before marking the task complete, confirm:

- [ ] All source files were read completely (no truncation)
- [ ] Written files were verified by reading back
- [ ] All Python 2 patterns have been converted
- [ ] File encodings are explicitly specified
- [ ] pathlib is used for all path operations
- [ ] pandas is used for data processing (where appropriate)
- [ ] requirements.txt includes all dependencies
- [ ] Script runs without errors
- [ ] Output matches expected format exactly
- [ ] Edge cases are handled (empty data, missing files)

## Output Validation

When the task specifies an expected output format, verify the output matches exactly:

1. Run the modernized script
2. Capture the output
3. Compare against expected format character-by-character if needed
4. Pay attention to:
   - Decimal precision in numerical output
   - Whitespace and formatting
   - Order of output items
   - Units and labels