🎵 Hip-Hop/Rap Profanity & Commercial Success Analysis (1990–2018)

📌 Project Overview

This project investigates the relationship between explicit lyrics and commercial success in Hip-Hop/Rap music during the digital era (1990–2018). Using the MusicOSet dataset, it applies text mining and statistical regression, in R, to test the popular assumption that "vulgarity sells."

Key research questions:

1. How has profanity density in Hip-Hop/Rap lyrics evolved over time (1990–2018)?
2. Is the "explicit" content label disproportionately applied to Hip-Hop/Rap compared to other genres?
3. Does profanity influence a Hip-Hop/Rap song's popularity?

---

📊 Key Findings

• The rise of profanity — Average profanity density in Hip-Hop/Rap climbed from around 1 word/song in 1990 to 14.7 words/song by 2018. Other genres, used here as a control group, stayed roughly flat over the same period.
• Genre concentration of "explicit" content — Hip-Hop/Rap accounts for 88% of all tracks tagged "explicit" in the dataset, despite being a much smaller share of the overall catalogue.
• Profanity ≠ success — Contrary to the "shock value sells" hypothesis, profanity count is weakly negatively correlated with commercial success in Hip-Hop/Rap (Pearson's r = -0.134). Other genres show essentially no relationship (r ≈ -0.01).

Full charts and write-up are in analysis.Rmd (knit to HTML for the complete, reproducible walkthrough) and in visualisation/.

---

🗂️ Repository Structure

├── R/
│   └── data_prep.R              # Shared data loading, cleaning & profanity-tagging logic
├── main.R                       # Core analysis: trend, genre composition, correlation charts
├── analysis.Rmd                 # Reproducible notebook — knits the full analysis to HTML
├── notebook-assets/             # CSS/JS for analysis.Rmd (styling, dark mode, skip link)
├── visualisation/
│   ├── more-charts-generate.R   # Additional/refined charts (composite dashboard)
│   ├── group_profanity_words.R  # Groups profane word variants under a root word
│   ├── images/                  # Generated chart outputs (from both main.R and this folder's script)
│   └── README.md                # Notes on the extended visualisation set
├── datasets/                    # MusicOSet tables + profanity lexicon (see below)
└── musicoset-hiphop-analysis.Rproj

main.R and visualisation/more-charts-generate.R both source R/data_prep.R for data loading and cleaning, so the pipeline logic lives in one place. analysis.Rmd walks through the same pipeline with narrative and can be knit end-to-end to reproduce every chart and statistic in this README.

---

📈 Method

1. Load MusicOSet metadata, popularity/chart, song-feature, and lyrics tables, plus a public profanity lexicon.
2. Clean lyrics — strip JSON/scraping artifacts, filter out malformed entries (e.g. non-lyric text mistakenly included as "lyrics").
3. Tag profanity — tokenise lyrics with tidytext and count matches against the profanity lexicon per song.
4. Classify genre — match songs to Hip-Hop/Rap using artist genre tags (main_genre/genres), against a regex covering rap, hip-hop, drill, grime, trap and related subgenres; everything else is the Other control group.
5. Restrict to 1990–2018 using each track's release year.
6. Analyse — temporal trend of profanity per genre group, share of "explicit"-tagged tracks by genre, and the correlation between profanity count and a chart-derived success score (cor.test).

---

▶️ Running the analysis

1. Open musicoset-hiphop-analysis.Rproj in RStudio (this sets the working directory to the project root, which all scripts assume).
2. Install dependencies if needed:

   install.packages(c("tidyverse", "tidytext", "ggplot2", "grid", "gridExtra",
                       "scales", "ggpubr", "ggrepel", "patchwork"))

3. Run main.R for the core charts, or visualisation/more-charts-generate.R for the extended/composite dashboard.
4. Or knit analysis.Rmd for the full reproducible write-up in one file.

---

Libraries used

• tidyverse (dplyr, readr, stringr) — data manipulation
• tidytext — tokenisation
• ggplot2 / patchwork / gridExtra / grid — data visualisation & layout
• ggrepel / ggpubr — plot annotation
• scales — formatting chart axes and labels

---

📁 Data source

MusicOSet (Silva et al., 2019) — a public dataset of ~20,000 songs with metadata, popularity/chart history, audio features, and lyrics. Profanity matching uses the badwords lexicon.

📄 Licence

Code in this repository is released under the MIT Licence. The MusicOSet dataset and the badwords lexicon retain their own original licences — see the respective linked sources.