Algorithmic
Design
Features

How age-based bans overlook the design choices that shape what young users see

Delaney McLaughlin Duke University MA in Applied Ethics & Policy AIPI 590

This page discusses self-harm and suicide. US: call or text 988. UK: Samaritans 116 123.

Molly Russell, 14

In November 2017, 14-year-old Molly Russell, from Harrow, north-west London, died from an act of self-harm while suffering from depression and the negative effects of online content.29,1

The coroner found the platforms’ algorithms produced “binge periods” of content, some “selected and provided without Molly requesting them.”29

16,300posts she saved, liked, or shared on Instagram in her last six months30,1

2,100of them related to depression, self-harm or suicide30,1

Each square is 100 posts.

Bans regulate access, not design.

Age-based bans regulate who can access social media, not how it is designed. Engagement-based ranking, the design choice driving harm to young users, remains largely unregulated, and explainable AI could make it inspectable before harm occurs rather than after.

Lens 1: Philosophical lens

Why engagement ranking is wrong

Four principles of biomedical ethics, applied to a ranked feed.25

  • Autonomy

    Ranking built to predict what a user will engage with, and keep supplying it, can undermine the conditions that make voluntary choice possible.25,26

  • Beneficence

    These systems are optimized for engagement, so any benefit to the user is incidental to their actual objective.25,3

  • Nonmaleficence

    When amplification of harmful content to vulnerable users is known, the resulting harm is foreseeable and preventable.25,31

  • Justice

    The costs fall hardest on minors, users in distress, and regions with little oversight, as Amnesty International found of Facebook’s role in Myanmar.25,24

Lens 2: Policy lens

Regulating access, not design

Five real measures, sorted by what they regulate. None fully regulates ranking.

Sort by
Measure What it regulates Regulates ranking design?
  1. Under-16 ban Australia Keeps under-16s off major platforms. The feed itself is untouched.5,34 Access Regulates ranking design: No
  2. SAFE for Kids Act New York Limits personalized feeds for minors from Jan. 25, 2027.6,45 Design Regulates ranking design: Partly
  3. Children’s Codes United Kingdom Recommends demoting harmful content for children.49,8 Design Regulates ranking design: Partly
  4. Digital Services Act European Union Disclosure, plus a non-profiling feed option.9,10 Transparency Regulates ranking design: Partly
  5. New Mexico v. Meta New Mexico $375M jury penalty plus a $567M abatement fund, but no design changes.17,59,65 Liability Regulates ranking design: No

Sorted by what each measure regulates.

“Require legislative or executive branch action.”

New Mexico’s judge, declining to order changes to infinite scroll and autoplay, August 2026.59,65

The thesis, in a court’s own words.

Lens 3: Technical lens

Inside a simulated feed

A ranker trained on 20,000 simulated impressions orders the feed of a simulated 14-year-old. Pick a post to see what pushed it up or down.28

47.4%

Distress-related posts were 15.7% of all simulated posts but 47.4% of the top 10% of rankings (38.0% without emotional intensity). The model was never told which posts were distress-related.28
  • All posts 15.7%
  • Top 10% of rankings 47.4%
  • Top 10%, without emotional intensity 38.0%

A simulated 14-year-old’s feed

Simulated feed, user and model. No real platform data.

Sad-edit video about feeling alone

Ranked #1. Score 0.964 = baseline 0.403 + these pushes.

  • User's past engagement with this topic +0.210
  • Avg. watch completion +0.071
  • Hours since posted +0.033
  • Reactions (popularity) +0.025
  • From an account the user follows −0.019
  • Comment rate +0.017
  • Emotional intensity of the post
  • User’s past engagement with this topic
  • Other signals

SHAP values: right pushes a post up, left pushes it down.69,28

Five tests of the explanations20,28

  • Fidelity Pass / Partial SHAP pass, LIME partial
  • Faithfulness Pass
  • Stability Pass In this case
  • Completeness Fail Platform-style reason
  • Human usefulness Fail Not tested with users

No change to the teen’s own history moved the top post out of her top three; only changing the content did.28,21

Lens 4: Public lens

What you’re told vs. what’s happening

TikTok and Meta have announced short reasons like this for why a post appears.61,18

Told

Why you’re seeing this

“This video is similar to content you've interacted with.”

A paraphrase of TikTok’s “Why this video” wording.19,61

Actually happening

35.1%explained by that reason

64.9%goes unmentioned

In the simulation, that reason explains only 35.1% of why the top post ranked first; the other 64.9%, including the post’s emotional intensity (its largest driver), goes unmentioned.28,19

Look before the harm

Bans decide who sees an engagement-ranked feed and leave untouched what that feed is built to maximize. Explainable AI can make the design inspectable before harm, but only if its explanations are tested against the people who rely on them.

Limitations The feed is simulated. Human usefulness was judged against the stated audience, not tested with users. Policy status is as of October 2026.

Sources & fact-check →