---
name: c-incremental-build-converter
description: Convert C/C++ project build.sh and test.sh scripts to support incremental builds. Use when fixing build scripts for re-execution after docker commit, converting to out-of-tree builds, or making scripts idempotent for repeated execution.
allowed-tools: Read, Grep, Glob, Bash, Edit, Write
---

# C/C++ Incremental Build Converter Skill

This skill converts C/C++ project build.sh and test.sh scripts to support incremental builds after docker commit.

## CRITICAL CONSTRAINTS

### 1. DO NOT Modify oss-fuzz Infrastructure Code

**NEVER modify these files:**
- `oss-fuzz/infra/helper.py`
- `oss-fuzz/infra/base-images/*`
- Any other oss-fuzz infrastructure code

**ONLY modify:**
- `oss-fuzz/projects/{project}/build.sh`
- `oss-fuzz/projects/{project}/test.sh`
- `oss-fuzz/projects/{project}/Dockerfile` (if absolutely necessary)

### 2. If oss-crs Test Framework Has Issues

If the incremental build test framework (`incremental_build_checker.py`) has bugs or limitations:
1. **Report the issue to the user** with clear explanation
2. **Suggest a fix** for the framework code
3. **DO NOT implement the fix yourself** - let the user decide

Example report format:
```
## oss-crs Framework Issue Detected

**File:** bug_fixing/src/oss_patch/inc_build_checker/incremental_build_checker.py
**Issue:** The source copy command destroys build artifacts
**Current behavior:** `rm -rf $SRC/nginx && cp -r ...`
**Expected behavior:** Should preserve objs/ directory
**Suggested fix:** Use rsync with --exclude objs instead of rm+cp
```

## Key Architecture: Separate $SRC for build.sh and test.sh

**IMPORTANT: $SRC is different between build.sh and test.sh.**

- build.sh runs in `/built-src` → `$SRC` = `/built-src`
- test.sh runs in `/test-src` → `$SRC` = `/test-src`

These are completely separate directories. This separation means:
1. **NO artifact cleanup needed** - Don't delete .o, .lo, .a, .la, .libs
2. **NO out-of-tree build needed** - In-source build is fine since $SRC is separate
3. **NO ASAN conflict** - test.sh builds in its own clean copy

### ALWAYS Use $SRC, NEVER Hardcode Paths

**CRITICAL: Always use `$SRC` variable, never hardcode `/src`, `/built-src`, or `/test-src`.**

```bash
# BAD - Hardcoded path breaks when $SRC changes
cd /src/project
cp -r /src/oniguruma modules/

# GOOD - Uses $SRC variable
cd $SRC/project
cp -r $SRC/oniguruma modules/
```

### AVOID $WORK - Use $SRC Subdirectories for All Build Artifacts

**CRITICAL: Avoid using $WORK for build-related directories. Use $SRC subdirectories instead.**

`$WORK` is shared between build.sh and test.sh, which causes problems:
1. ASAN-instrumented libraries from build.sh conflict with test.sh
2. CMake paths recorded in $WORK become invalid when $SRC changes
3. Build artifacts from different environments can corrupt each other

**Always use `$SRC` subdirectories for:**
- Build directories (out-of-source builds)
- Install prefixes (for cmake/configure --prefix)
- Dependency installations

```bash
# BAD - Uses $WORK which is shared
BUILD_DIR="$WORK/build"
cmake "$SRC/project" -DCMAKE_INSTALL_PREFIX=$WORK

# GOOD - Uses $SRC subdirectories
BUILD_DIR="$SRC/project_build"
INSTALL_DIR="$SRC/project_install"
mkdir -p "$BUILD_DIR" "$INSTALL_DIR"
cd "$BUILD_DIR"
cmake "$SRC/project" -DCMAKE_INSTALL_PREFIX="$INSTALL_DIR"
```

**In test.sh, also use $SRC subdirectories:**

```bash
# BAD in test.sh - Uses shared $WORK with ASAN artifacts
./configure --prefix="$WORK"
# Error: undefined reference to __asan_*

# GOOD in test.sh - Uses separate prefix under $SRC
TEST_PREFIX="$SRC/test_deps"
mkdir -p "$TEST_PREFIX/lib" "$TEST_PREFIX/include"

if [ ! -f "$TEST_PREFIX/lib/libz.a" ]; then
    pushd "$SRC/zlib"
    ./configure --static --prefix="$TEST_PREFIX"
    make -j$(nproc) CFLAGS="-fPIC"  # No ASAN flags
    make install
    popd
fi
```

### test.sh Does NOT Need Sanitizers - Disable Them Explicitly

**IMPORTANT: test.sh runs unit tests, NOT fuzzing. Sanitizers are NOT needed and cause conflicts.**

During incremental build testing, build.sh compiles with ASAN/sanitizer flags. When test.sh runs in the SAME directory, the object files from build.sh are ASAN-instrumented. If test.sh tries to link against these objects without ASAN runtime, you get undefined reference errors.

### build.sh and test.sh Artifacts Should NOT Be Shared

**CRITICAL: Build artifacts from build.sh should NOT be reused in test.sh.**

During incremental build testing:
1. build.sh runs with ASAN flags → creates ASAN-instrumented `.o` files
2. test.sh runs in the SAME `$SRC` directory
3. If test.sh tries to use existing `.o` files, linking fails

**The artifacts are incompatible because:**
- build.sh: Compiles with `-fsanitize=address -fsanitize-address-use-after-scope -fsanitize=fuzzer-no-link`
- test.sh: Needs to link without sanitizer runtime


**DO NOT use `|| true` to ignore link failures** - this hides the real problem. Fix the root cause by ensuring clean separation of build artifacts.

## Critical Requirements

### 1. Scripts MUST Work on BOTH First Run AND Repeated Runs

**NEVER** assume a script will only run once. Scripts must be idempotent.

```bash
# BAD - Runs configure every time
autoreconf -f -i
./configure

# GOOD - Only configure on first run
if [ ! -f Makefile ]; then
    autoreconf -f -i
    ./configure
fi
make -j$(nproc)
```

### 2. DO NOT Delete Build Artifacts

Since $SRC is separate, there's no need to clean artifacts. Deleting them defeats incremental build.

```bash
# BAD - Destroys incremental build benefits
find . -name "*.o" -delete
find . -name "*.a" -delete
rm -rf .libs

# GOOD - Just build, artifacts from previous run will speed things up
if [ ! -f Makefile ]; then
    ./configure
fi
make -j$(nproc)
```

### 3. Conditional Configure/Autoreconf

Only run configure-related commands on first execution.

```bash
# Pattern: Check for Makefile existence
if [ ! -f Makefile ]; then
    autoreconf -fi       # or ./bootstrap
    ./configure --options
fi
make -j$(nproc)
make check
```

### 4. Use `cp` Instead of `mv`

`mv` removes the source file, causing failures on repeated runs.

```bash
# BAD - Fails on second run
mv scripts/config.temp scripts/config

# GOOD - Preserves source file
[ -f scripts/config.temp ] && cp scripts/config.temp scripts/config || true
```

### 5. Use Guards for Dependencies

```bash
# Build dependency only if not already built
if [ ! -f "$WORK/lib/libz.a" ]; then
    pushd "$SRC/zlib"
    ./configure --static --prefix="$WORK"
    make -j$(nproc)
    make install
    popd
fi
```

### 6. Handle Failing Tests

Remove failing test files directly instead of using TESTS variable (which can propagate to submodules).

```bash
# BAD - TESTS variable propagates to submodules
make check TESTS="tests/mantest tests/jqtest"

# GOOD - Remove failing test files directly
rm -f tests/shtest
rm -f tests/multiple.*
sed -i '/multiple/d' tests/Makefile.am
make check
```

### 7. Copy Dependencies If Not Present (Conditional, Not Delete-and-Copy)

```bash
# BAD - Deletes and recopies every time, breaks incremental build
rm -rf modules/oniguruma
rsync -a $SRC/oniguruma modules/

# GOOD - Only copy if not present
if [ ! -d modules/oniguruma ]; then
    mkdir -p modules
    cp -r $SRC/oniguruma modules/
fi
```

### 8. Conditional Dict/Corpus File Copy

Files like dictionaries and seed corpus may not exist in all builds.

```bash
# BAD - Fails if file doesn't exist
cp $SRC/libtiff/contrib/oss-fuzz/tiff.dict $OUT/

# GOOD - Conditional copy with fallback
[ -f $SRC/libtiff/contrib/oss-fuzz/tiff.dict ] && cp $SRC/libtiff/contrib/oss-fuzz/tiff.dict $OUT/ || true
find . -name "*.tif" | xargs zip $OUT/seed_corpus.zip || true
```

### 9. Use `make -k check || true` for Potentially Failing Tests

Some tests may fail in fuzzing environment. Use `-k` to continue and `|| true` to not fail the script.

```bash
# BAD - Stops on first test failure
make check

# GOOD - Continue through failures
make -k check || true

echo "Tests completed"
```

### 10. Remove Tests from Makefile.am BEFORE configure

When removing failing tests, modify Makefile.am BEFORE running autoreconf/configure.

```bash
# BAD - Removes test file but Makefile still references it
rm -f tests/shtest
autoreconf -fi
./configure
make check  # Error: No rule to make target 'tests/shtest'

# GOOD - Remove from Makefile.am before configure
sed -i 's| tests/shtest||g' Makefile.am
autoreconf -fi
./configure
make check  # Works
```

### 11. Rewrite Instead of Sourcing Internal build.sh

If the project's source contains its own build.sh (e.g., `contrib/oss-fuzz/build.sh`), it may use `mv` or other non-idempotent operations. Rewrite the logic in the project's build.sh instead.

```bash
# BAD - Sources internal build.sh that uses mv
source $SRC/project/contrib/oss-fuzz/build.sh

# GOOD - Rewrite the logic with cp instead of mv
if [ ! -f "$WORK/lib/libjbig.a" ]; then
    pushd "$SRC/jbigkit"
    make lib
    # Use cp instead of mv for incremental build support
    cp "$SRC"/jbigkit/libjbig/*.a "$WORK/lib/"
    cp "$SRC"/jbigkit/libjbig/*.h "$WORK/include/"
    popd
fi
```

### 12. Guard `sed -i` Insertions with Pattern Checks

When using `sed -i` to insert content, check BOTH the original pattern exists AND the new content doesn't exist yet.

```bash
# BAD - Only checks if new content exists (can't distinguish original vs added)
if ! grep -q '%destructor' file.y; then
    sed -i '/^%%$/i%destructor { free ($$); } ID' file.y
fi

# GOOD - Check original pattern exists AND new content not yet added
if grep -q '^%%$' file.y && ! grep -q '%destructor' file.y; then
    sed -i '/^%%$/i%destructor { free ($$); } ID' file.y
fi
```

### 13. Guard Append Operations (`>>`)

Don't blindly use `>` (overwrite) or `>>` (append). Check if content already exists.

```bash
# BAD - Overwrites every time (inefficient, may lose state)
sed 's/main/old_main/' file.c > header.h

# BAD - Appends every time (file grows on each run)
sed 's/main/old_main/' file.c >> header.h

# GOOD - Only append if content not already present
if ! grep -q 'old_main' header.h 2>/dev/null; then
    sed 's/main/old_main/' file.c >> header.h
fi
```

### 14. CMake/Ninja Build Directory Must Use $SRC-Relative Path

When `$SRC` changes between build.sh and test.sh, CMake/Ninja will fail with "manifest dirty" errors if build directory is in `$WORK`.

```bash
# BAD - $WORK is shared, CMake source path mismatch causes ninja errors
BUILD_DIR="$WORK/build"
cd "$BUILD_DIR"
cmake $SRC/project  # Source path recorded as /built-src/project
# On test.sh: ninja fails because $SRC is now /test-src/project

# GOOD - Build directory relative to $SRC, each environment independent
BUILD_DIR="$SRC/project_build"
mkdir -p "$BUILD_DIR"
cd "$BUILD_DIR"
if [ ! -f build.ninja ]; then
    cmake -GNinja $SRC/project
fi
ninja
```

### 15. Disable ccache for Bazel Builds

Bazel has its own caching mechanism. ccache interferes with Bazel's compiler wrappers.

```bash
# BAD - ccache causes "invalid option" errors with Bazel
# Error: /usr/local/bin/ccache: invalid option -- 'U'
bazel build //...

# GOOD - Remove ccache from PATH before Bazel builds
export PATH=$(echo "$PATH" | sed 's|/ccache/bin:||g; s|:/ccache/bin||g')
unset CC CXX
export CC=clang
export CXX=clang++
bazel build //...
```

## Standard test.sh Template

**IMPORTANT: Execution Environment**
- `$SRC` is **already set** (no need for `: "${SRC:=/src}"`)
- Working directory is **already `$SRC/{project}`** (no need for `cd $SRC/{project}`)

```bash
#!/bin/bash
set -ex

# Build (only configure on first run)
if [ ! -f Makefile ]; then
    autoreconf -fi  # or ./bootstrap, or skip if not autotools
    ./configure --options
fi
make -j$(nproc)

# Run tests (use -k to continue on failures, || true to not fail script)
make -k check || true

echo "Tests completed"
```

## Examples by Build System

### Autotools Project

```bash
#!/bin/bash
set -ex

if [ ! -f Makefile ]; then
    autoreconf -fi
    ./configure --enable-static
fi
make -j$(nproc)
make -k check || true
```

### CMake Project

```bash
#!/bin/bash
set -ex

if [ ! -f Makefile ]; then
    cmake . -DCMAKE_INSTALL_PREFIX=$WORK -DBUILD_SHARED_LIBS=off
fi
make -j$(nproc)
make test
```

### Project with Dependencies

```bash
#!/bin/bash
set -ex

# Build zlib (only on first run)
if [ ! -f "$WORK/lib/libz.a" ]; then
    pushd "$SRC/zlib"
    ./configure --static --prefix="$WORK"
    make -j$(nproc)
    make install
    popd
fi

# Build main project
if [ ! -f Makefile ]; then
    cmake . -DCMAKE_INSTALL_PREFIX=$WORK
fi
make -j$(nproc)
make test
```

### Project with Submodules (e.g., jq with oniguruma)

```bash
#!/bin/bash
set -ex

# Copy submodule if not present (conditional, not delete-and-copy)
if [ ! -d modules/oniguruma ]; then
    mkdir -p modules
    cp -r $SRC/oniguruma modules/
fi

# Remove failing tests from Makefile.am BEFORE configure
sed -i 's| tests/shtest||g' Makefile.am

if [ ! -f Makefile ]; then
    autoreconf -fi
    ./configure --with-oniguruma=builtin
fi
make -j$(nproc)

# Run tests (some may fail in fuzzing environment)
make -k check || true
```

## Common Errors and Fixes

| Error | Cause | Fix |
|-------|-------|-----|
| `undefined reference to __asan_*` | $WORK shared with ASAN artifacts from build.sh | Use separate prefix like `$SRC/test_deps` in test.sh |
| `Makefile:xxx: No rule to make target` | Partial build state | Use `if [ ! -f Makefile ]` guard around configure |
| `No rule to make target 'tests/shtest'` | Test removed but still in Makefile.am | Use `sed -i 's| tests/shtest||g' Makefile.am` BEFORE configure |
| `source directory already configured` | Previous configure run | Check for Makefile before configure |
| `mv: cannot stat: No such file` | File moved in previous run | Use `cp` with guard |
| `failed to create tests/*.trs` | TESTS= propagates to submodules | Remove failing test files directly |
| `No rule to make target 'jbig.h'` | Original build.sh used `mv` to move files | Rewrite to use `cp` instead |
| `tiff.dict: No such file` | Dict/corpus file doesn't exist | Use conditional copy: `[ -f file ] && cp file dest \|\| true` |
| `No rule to make target 'all'` in submodule | `rm -rf && rsync` deleted submodule's Makefile | Use conditional copy: `if [ ! -d dir ]; then cp; fi` |
| `%destructor redeclaration` / duplicate content | `sed -i` insertion runs every time | Guard with pattern check: `if grep -q 'pattern' && ! grep -q 'new_content'` |
| File grows on each build | `>>` append runs every time | Guard append: `if ! grep -q 'content' file; then ... >> file; fi` |
| `ninja: manifest 'build.ninja' still dirty` | CMake source path mismatch ($SRC changed) | Use `$SRC`-relative build dir instead of `$WORK` |
| `ccache: invalid option -- 'U'` | ccache interferes with Bazel | Remove ccache from PATH: `export PATH=$(echo "$PATH" \| sed 's\|/ccache/bin:\|\|g')` |

## Testing Incremental Builds

**CRITICAL: Use the `test-inc-build` subagent via Task tool to prevent context limit errors.**

The subagent runs tests, analyzes logs internally, and returns only essential summary information.

### Single Project Test

```
Task tool:
  subagent_type: "test-inc-build"
  description: "Test incremental build"
  prompt: "Run incremental build test for {project_name} with oss-fuzz path ../oss-fuzz"
```

### AIXCC Projects: Use Full Path Prefix

**IMPORTANT: For AIXCC projects, use the `aixcc/c/` or `aixcc/jvm/` prefix in the project name.**

Examples:
- `aixcc/c/afc-sqlite3-delta-01`
- `aixcc/c/atlanta-libjpeg-full-01`
- `aixcc/jvm/afc-jenkins-delta-01`

### Parallel Testing Multiple Projects

Use Task tool with `run_in_background: true` in a single message:

```
Task tool (run_in_background: true):
  subagent_type: "test-inc-build"
  prompt: "Run incremental build test for aixcc/c/afc-libexif-delta-01 with oss-fuzz path ../oss-fuzz"

Task tool (run_in_background: true):
  subagent_type: "test-inc-build"
  prompt: "Run incremental build test for aixcc/c/afc-libexif-delta-02 with oss-fuzz path ../oss-fuzz"

Then use TaskOutput to retrieve results from each agent.
```

### What the Test Does

1. Runs build.sh (first run)
2. Runs docker commit to save state
3. Runs build.sh again (incremental run)
4. Compares build times and reports reduction percentage

### Subagent Return Format

**On Success:**
```
## Result: SUCCESS
**Project:** {project_name}
**Build Time Reduction:** {percentage}%
**Summary:** Incremental build working correctly.
```

**On Failure:**
```
## Result: FAILED
**Project:** {project_name}
**Error Type:** {category}
**Error Message:** {exact error, max 10 lines}
**Suggested Fix:** {brief fix}
```

**Success criteria:**
- Both runs complete without errors
- Build time reduction is reported (higher % = better incremental build support)

## Checklist

- [ ] Uses `$SRC` variable, NOT hardcoded paths like `/src` or `/built-src`
- [ ] Uses `if [ ! -f Makefile ]` guard around configure/autoreconf
- [ ] Does NOT delete build artifacts (.o, .lo, .a, .la, .libs)
- [ ] Uses `cp` instead of `mv` where applicable
- [ ] Failing tests removed from Makefile.am BEFORE configure (not just file deletion)
- [ ] Dependencies have existence guards
- [ ] Directory copies are conditional (`if [ ! -d ]`), not delete-and-copy
- [ ] Uses `$SRC` subdirectories instead of `$WORK` for build dirs and install prefixes
- [ ] test.sh uses separate prefix (e.g., `$SRC/test_deps`) if dependencies needed
- [ ] Dict/corpus copies use conditional: `[ -f file ] && cp ... || true`
- [ ] Tests use `make -k check || true` if some may fail
- [ ] Internal build.sh rewritten if it uses `mv` or non-idempotent operations
- [ ] `sed -i` insertions guarded with pattern + content check
- [ ] Append operations (`>>`) guarded with content existence check
- [ ] CMake/Ninja build directories use `$SRC`-relative paths (not `$WORK`)
- [ ] Bazel builds have ccache removed from PATH
- [ ] Works on first run AND repeated runs