--- name: "echoes-loop-diagnosing-risks" description: "Diagnose and mitigate feedback-loop risks (bias amplification, hallucination propagation, exposure polarization) in LLM-powered recommender systems using a role-aware, phase-wise diagnostic framework. Use when: 'audit my recommendation pipeline for bias', 'check feedback loop risks in my LLM recommender', 'diagnose hallucination propagation in recommendations', 'build a feedback-loop simulation for my recommender', 'trace popularity bias through my recommendation cycles', 'add risk monitoring to my LLM-based ranking system'." --- # Echoes in the Loop: Diagnosing Risks in LLM-Powered Recommender Systems This skill enables Claude to systematically diagnose, simulate, and mitigate the compounding risks that emerge when LLMs operate within recommender system feedback loops. Based on the role-aware, phase-wise diagnostic framework from Park, Lee & Lee (2026), it provides a structured methodology for tracing how LLM-specific risks — popularity bias amplification, hallucination-induced spurious signals, and self-reinforcing exposure polarization — emerge from specific LLM functional roles, manifest in ranking outcomes, and accumulate across repeated recommendation cycles. Claude applies this framework to audit existing pipelines, build simulation harnesses, and instrument production systems with risk-aware monitoring. ## When to Use - When the user is building or maintaining a recommender system that uses LLMs for any stage (content generation, user profiling, ranking, re-ranking, explanation) - When the user asks to audit a recommendation pipeline for bias, fairness, or hallucination risks - When the user wants to simulate how recommendation quality degrades over multiple feedback cycles - When the user needs to instrument an LLM-based recommender with risk monitoring and alerting - When the user is designing a new LLM-powered recommendation feature and wants to anticipate systemic risks before deployment - When the user observes narrowing recommendation diversity, popularity concentration, or echo-chamber effects in their system - When the user asks about feedback loop dynamics in any AI system that consumes its own outputs over time ## Key Technique ### The Role-Aware, Phase-Wise Diagnostic Framework Traditional recommender system audits evaluate a single snapshot of recommendations. This framework recognizes that LLMs introduce *qualitatively different* risks depending on their **functional role** in the pipeline, and that these risks **compound through feedback loops** across multiple recommendation cycles (phases). The framework identifies three primary LLM functional roles, each with distinct risk profiles: 1. **Data Augmentation Role** — LLMs generate synthetic user reviews, item descriptions, or features to fill data gaps. Risk: hallucinated attributes (e.g., fabricating that a niche book is a "bestseller") inject spurious signals that downstream models treat as ground truth. Over cycles, these hallucinations self-reinforce as the system recommends items based on fabricated properties, collects implicit feedback on those recommendations, and feeds that feedback back into training. 2. **Profiling Role** — LLMs summarize user preferences, extract interests from interaction histories, or generate user embeddings. Risk: LLMs inherit and amplify popularity bias from training data, producing profiles that over-index on mainstream preferences. Over cycles, users receive increasingly homogeneous recommendations, their interaction histories converge, and subsequent profiles become even more biased — a classic positive feedback loop. 3. **Decision-Making Role** — LLMs directly score, rank, or select items for recommendation. Risk: LLMs exhibit position bias, verbosity bias, and anchoring effects that distort rankings. Over cycles, items that benefit from these biases accumulate more exposure, more interactions, and thus more evidence to justify continued promotion — creating exposure polarization where popular items monopolize attention. ### Three-Level Diagnostic Measurement The framework measures risk at three levels, each corresponding to a progressively wider blast radius: (a) **Content Level** — evaluating the LLM-generated artifacts themselves for hallucination rates, factual consistency, and distributional skew; (b) **Ranking Level** — measuring how content-level risks translate into ranking distortions such as popularity bias in top-K lists, diversity collapse, and fairness violations; (c) **Ecosystem Level** — tracking system-wide dynamics over multiple cycles including Gini concentration of item exposure, user preference homogenization, and self-reinforcing feedback patterns. ## Step-by-Step Workflow ### 1. Map LLM Roles in the Pipeline Enumerate every point where an LLM is invoked in the recommender pipeline. For each invocation, classify it as **data augmentation**, **profiling**, or **decision-making**. Document the input/output contract: what data flows in, what the LLM produces, and where its output is consumed downstream. ### 2. Identify Risk Vectors per Role For each LLM role identified in Step 1, enumerate the applicable risk categories: - Data Augmentation → hallucination (factual fabrication, attribute invention), distributional skew (over-representing popular item characteristics) - Profiling → popularity bias amplification, preference homogenization, stereotyping - Decision-Making → position bias, verbosity bias, anchoring, exposure concentration ### 3. Instrument Content-Level Metrics Add measurement points at each LLM output to capture: - **Hallucination rate**: Compare LLM-generated attributes against a ground-truth catalog; flag attributes with no supporting evidence - **Distributional skew**: Measure KL-divergence between the distribution of generated attributes and the true item attribute distribution - **Novelty ratio**: Track the fraction of generated content that introduces information not present in the input ### 4. Instrument Ranking-Level Metrics Add measurement points after the ranking stage to capture: - **Popularity bias**: Average Recommendation Popularity (ARP), popularity stratified recall (long-tail vs. head items) - **Diversity**: Intra-list diversity (ILD), coverage (fraction of catalog recommended over a window) - **Fairness**: Exposure parity across item groups (provider fairness), demographic parity across user groups ### 5. Build a Feedback-Loop Simulation Harness Construct a simulation that runs the recommendation pipeline for N cycles: - Initialize with real user interaction histories - At each cycle: run the full pipeline (LLM augmentation → profiling → ranking), simulate user interactions with the recommended items (using click models or heuristic acceptance), fold the simulated interactions back into the training/input data - Record all content-level, ranking-level, and ecosystem-level metrics at each cycle ### 6. Instrument Ecosystem-Level Metrics Across simulation cycles, track: - **Gini coefficient** of item exposure: measures concentration (1.0 = one item gets all exposure) - **User preference entropy**: average entropy of user profiles over time (declining = homogenization) - **Feedback loop gain**: rate of change of popularity bias or Gini coefficient per cycle (>1.0 = amplifying) - **Echo chamber index**: fraction of users whose top-K recommendations overlap by >80% with their previous cycle ### 7. Run the Diagnostic and Identify Amplification Patterns Execute the simulation for 10-20 cycles. Plot each metric over time. Look for: - Monotonically increasing popularity concentration → popularity bias amplification - Declining user preference entropy → preference homogenization / echo chambers - Hallucination persistence or growth → spurious signal propagation - Super-linear metric growth → positive feedback loop (the "echo") ### 8. Trace Root Causes to Specific LLM Roles When amplification is detected, trace backward through the pipeline to the originating LLM role. Compare metrics with and without each LLM component (ablation) to isolate the contribution of each role to the observed risk. ### 9. Apply Targeted Mitigations Based on the diagnosed role and risk: - **Data Augmentation hallucination** → Add factual grounding checks, constrain generation to catalog-verified attributes, use retrieval-augmented generation - **Profiling bias** → Inject diversity priors into profile generation prompts, calibrate LLM outputs against population-level statistics - **Decision-making concentration** → Apply re-ranking with exposure fairness constraints, use position-debiased prompting, add exploration budget (epsilon-greedy or Thompson sampling) ### 10. Establish Continuous Monitoring Deploy the content-level and ranking-level metrics as production monitors. Set alerts for: - Hallucination rate exceeding baseline by >2 standard deviations - Gini coefficient crossing a defined threshold - Feedback loop gain exceeding 1.0 over a rolling window of cycles ## Concrete Examples **Example 1: Auditing an LLM-augmented product recommender** User: "We use GPT-4 to generate product descriptions for items missing catalog data, then feed those into our collaborative filtering model. Lately our recommendations seem very same-y. Can you help diagnose what's happening?" Approach: 1. Map the pipeline: LLM plays a **data augmentation** role — generating product descriptions consumed by the CF model 2. Instrument content-level metrics on the generated descriptions: ```python # content_audit.py import json from collections import Counter def audit_generated_descriptions(generated: list[dict], catalog: list[dict]): """Compare LLM-generated product attributes against ground-truth catalog.""" hallucination_flags = [] popularity_tokens = Counter() catalog_attrs = {item["id"]: set(item.get("attributes", [])) for item in catalog} for item in generated: item_id = item["id"] gen_attrs = set(item.get("generated_attributes", [])) true_attrs = catalog_attrs.get(item_id, set()) # Hallucination: generated attributes not in catalog hallucinated = gen_attrs - true_attrs hallucination_flags.append({ "item_id": item_id, "hallucinated_attributes": list(hallucinated), "hallucination_rate": len(hallucinated) / max(len(gen_attrs), 1), }) # Track token frequency to detect distributional skew for attr in gen_attrs: popularity_tokens[attr] += 1 avg_hallucination_rate = sum( f["hallucination_rate"] for f in hallucination_flags ) / len(hallucination_flags) # Compute Gini of attribute frequency distribution freqs = sorted(popularity_tokens.values()) n = len(freqs) gini = (2 * sum((i + 1) * f for i, f in enumerate(freqs)) / (n * sum(freqs))) - (n + 1) / n if n > 0 and sum(freqs) > 0 else 0 return { "avg_hallucination_rate": avg_hallucination_rate, "attribute_gini": gini, "top_hallucinated": sorted(hallucination_flags, key=lambda x: x["hallucination_rate"], reverse=True)[:10], } ``` 3. Build a feedback-loop simulation: ```python # feedback_loop_sim.py def simulate_feedback_loop(pipeline, users, items, n_cycles=15): """Simulate recommendation cycles and track risk metrics.""" metrics_over_time = [] for cycle in range(n_cycles): # Step A: LLM augments item data augmented_items = pipeline.augment(items) # Step B: Generate recommendations recommendations = pipeline.recommend(users, augmented_items, top_k=10) # Step C: Simulate user interactions (click model) interactions = simulate_clicks(users, recommendations, position_bias=True) # Step D: Measure metrics at all three levels content_metrics = measure_content_level(augmented_items, items) ranking_metrics = measure_ranking_level(recommendations, items) ecosystem_metrics = measure_ecosystem_level( recommendations, users, metrics_over_time ) metrics_over_time.append({ "cycle": cycle, **content_metrics, **ranking_metrics, **ecosystem_metrics, }) # Step E: Fold interactions back into training data pipeline.update(interactions) items = augmented_items # Generated data becomes next cycle's input return metrics_over_time ``` Output: A diagnostic report showing hallucination rate climbing from 8% to 23% over 15 cycles, with Gini coefficient of item exposure rising from 0.45 to 0.72, confirming that fabricated attributes on popular items create a self-reinforcing loop. --- **Example 2: Adding risk monitoring to an LLM-based news recommender** User: "Our news app uses an LLM to summarize user reading histories into interest profiles, then ranks articles by profile relevance. How do I monitor for echo chamber effects?" Approach: 1. Map the pipeline: LLM plays a **profiling** role — summarizing histories into interest profiles 2. Instrument ecosystem-level metrics: ```python # echo_chamber_monitor.py import numpy as np from scipy.stats import entropy def compute_echo_chamber_metrics( user_profiles: dict[str, list[str]], recommendations: dict[str, list[str]], previous_recommendations: dict[str, list[str]], ): """Detect preference homogenization and echo chamber formation.""" # 1. User preference entropy (are profiles converging?) all_topics = set() for topics in user_profiles.values(): all_topics.update(topics) topic_list = sorted(all_topics) entropies = [] for user_id, topics in user_profiles.items(): topic_counts = np.array([topics.count(t) for t in topic_list], dtype=float) if topic_counts.sum() > 0: topic_dist = topic_counts / topic_counts.sum() entropies.append(entropy(topic_dist)) avg_entropy = np.mean(entropies) # 2. Recommendation overlap with previous cycle overlaps = [] for user_id in recommendations: if user_id in previous_recommendations: current = set(recommendations[user_id][:10]) previous = set(previous_recommendations[user_id][:10]) overlap = len(current & previous) / max(len(current), 1) overlaps.append(overlap) avg_overlap = np.mean(overlaps) if overlaps else 0 # 3. Cross-user recommendation similarity (homogenization) user_ids = list(recommendations.keys()) pairwise_overlaps = [] for i in range(min(len(user_ids), 200)): for j in range(i + 1, min(len(user_ids), 200)): set_i = set(recommendations[user_ids[i]][:10]) set_j = set(recommendations[user_ids[j]][:10]) pairwise_overlaps.append( len(set_i & set_j) / max(len(set_i | set_j), 1) ) cross_user_similarity = np.mean(pairwise_overlaps) if pairwise_overlaps else 0 return { "avg_profile_entropy": avg_entropy, "avg_cycle_overlap": avg_overlap, "cross_user_similarity": cross_user_similarity, "echo_chamber_alert": avg_overlap > 0.8 or avg_entropy < 1.0, } ``` 3. Set up alerting thresholds: ```python # alerts.py RISK_THRESHOLDS = { "avg_profile_entropy": {"warn": 1.5, "critical": 1.0, "direction": "below"}, "avg_cycle_overlap": {"warn": 0.6, "critical": 0.8, "direction": "above"}, "cross_user_similarity": {"warn": 0.3, "critical": 0.5, "direction": "above"}, "feedback_loop_gain": {"warn": 1.0, "critical": 1.5, "direction": "above"}, } def check_alerts(metrics: dict) -> list[dict]: alerts = [] for metric, thresholds in RISK_THRESHOLDS.items(): value = metrics.get(metric) if value is None: continue if thresholds["direction"] == "above": if value >= thresholds["critical"]: alerts.append({"metric": metric, "value": value, "level": "CRITICAL"}) elif value >= thresholds["warn"]: alerts.append({"metric": metric, "value": value, "level": "WARNING"}) else: if value <= thresholds["critical"]: alerts.append({"metric": metric, "value": value, "level": "CRITICAL"}) elif value <= thresholds["warn"]: alerts.append({"metric": metric, "value": value, "level": "WARNING"}) return alerts ``` Output: A monitoring dashboard that tracks profile entropy, cycle-over-cycle overlap, and cross-user homogenization, alerting when echo chamber indicators exceed thresholds. --- **Example 3: Designing a new LLM re-ranker with built-in risk controls** User: "I want to use an LLM to re-rank search results for our e-commerce site. How do I avoid the feedback loop problems?" Approach: 1. Map the role: LLM will serve a **decision-making** role — directly determining final ranking 2. Design with risk-aware constraints from the start: ```python # risk_aware_reranker.py import random def risk_aware_llm_rerank( query: str, candidates: list[dict], llm_client, exploration_rate: float = 0.1, max_popularity_ratio: float = 0.5, ): """LLM re-ranker with built-in feedback loop mitigations.""" # Mitigation 1: Shuffle candidates before LLM scoring to reduce position bias shuffled = candidates.copy() random.shuffle(shuffled) # Mitigation 2: Prompt design that counters popularity anchoring prompt = f"""Score the relevance of each product to the query: "{query}" Rate ONLY based on feature match to the query. Ignore popularity, review count, or brand recognition. A niche product that perfectly matches the query should score higher than a popular product that partially matches. Products: {format_products(shuffled)} Return scores as JSON: [{{"id": "...", "score": 0.0-1.0}}]""" scores = llm_client.generate(prompt) scored = sorted(scores, key=lambda x: x["score"], reverse=True) # Mitigation 3: Exposure fairness constraint head_items = [s for s in scored if s.get("popularity_tier") == "head"] head_count = sum(1 for s in scored[:10] if s.get("popularity_tier") == "head") if head_count / 10 > max_popularity_ratio: scored = rebalance_exposure(scored, max_popularity_ratio) # Mitigation 4: Exploration budget — inject random long-tail items final = scored[:10] n_explore = max(1, int(len(final) * exploration_rate)) long_tail = [c for c in candidates if c.get("popularity_tier") == "tail"] if long_tail: for i in range(n_explore): pos = random.randint(5, 9) # Insert in lower half final[pos] = random.choice(long_tail) return final ``` Output: A re-ranker that proactively mitigates position bias (shuffling), popularity anchoring (prompt design), exposure concentration (fairness constraint), and feedback loops (exploration budget). ## Best Practices **Do:** - Always map LLM roles before diagnosing — the same risk manifests differently depending on whether the LLM augments data, builds profiles, or makes decisions - Simulate at least 10 feedback cycles; single-snapshot audits miss compounding effects entirely - Measure at all three levels (content, ranking, ecosystem) — content-level hallucinations may appear harmless in isolation but cause ranking-level distortions that become ecosystem-level crises - Use ablation studies (disable one LLM role at a time) to isolate which role drives observed risk amplification **Avoid:** - Do not assume that low hallucination rates are safe — even 2-3% hallucination rates compound rapidly through feedback loops over 10+ cycles - Do not treat recommendation diversity metrics as sufficient on their own — a system can show adequate diversity while still homogenizing user preferences across the population - Do not apply uniform mitigations across all LLM roles — a re-ranking fairness constraint does nothing for data augmentation hallucinations; match the mitigation to the diagnosed role and risk - Do not skip the feedback-loop simulation and rely only on static A/B tests — feedback loop risks are invisible in short-horizon evaluations ## Error Handling - **Incomplete ground truth catalog**: When auditing hallucination in data augmentation, missing catalog data leads to false positives. Mitigate by using high-confidence catalog subsets and reporting hallucination rates as ranges with confidence intervals. - **Click model assumptions in simulation**: Simulated user feedback depends heavily on the click model chosen (position-biased, cascade, etc.). Run simulations with 2-3 different click models and report results under each; if risk amplification appears under all models, the finding is robust. - **LLM non-determinism**: The same prompt may produce different outputs across runs, adding noise to metrics. Use fixed random seeds where possible, average over 3-5 runs per cycle, and report variance alongside means. - **Scale limitations**: Full feedback-loop simulation with LLM calls at every cycle is expensive. For initial diagnostics, use cached or mocked LLM outputs for intermediate cycles, running full LLM inference at cycles 0, 5, 10, 15 to capture the trajectory. ## Limitations - The framework assumes a closed feedback loop where the system's outputs directly influence future inputs. Recommender systems with strong external signals (e.g., social sharing, search-driven discovery) may have their feedback loops partially dampened, reducing the diagnostic signal. - Simulation fidelity depends on the click model approximating real user behavior. Actual user responses to recommendations involve complex factors (mood, context, social influence) that no click model fully captures. - The framework diagnoses risks and quantifies their trajectory but does not automatically determine the optimal mitigation strength — that requires balancing relevance against fairness objectives, which is domain-specific. - Ecosystem-level metrics require sufficient user and item volume to be statistically meaningful. Systems with fewer than ~1,000 active users or ~500 items may not produce reliable Gini or entropy measurements. ## Reference Park, D., Lee, D., & Lee, Y.-C. (2026). *Echoes in the Loop: Diagnosing Risks in LLM-Powered Recommender Systems under Feedback Loops*. arXiv:2602.07442. https://arxiv.org/abs/2602.07442v1 Key insight to look for: The role-aware decomposition (data augmentation / profiling / decision-making) combined with three-level measurement (content / ranking / ecosystem) across temporal phases, which reveals compounding risks invisible to single-snapshot evaluations.