Exploring Ethics of AI and Algorithmic Bias in Gemini's Songwriting

Maya Lorigiola

Introduction

Recently, when I logged onto Gemini for the usual everyday productivity tasks, I noticed a new “Create Music” feature. After some research I understood it was a brand-new feature added in Feb 2026. Powered by DeepMind's Lyria 3 music-generation model (TechCrunch, 2026; Google, 2026a), users simply have to describe a song in plain language and Gemini returns a 30-second track with vocals, lyrics, and AI-generated cover art.

As someone passionate about songwriting myself, this feature sparked my curiosity to test how creative AI can really be, and whether a model like this can navigate the ethical terrain it sits on. I wanted to explore algorithmic bias and intellectual property in AI-generated music. To do that, I asked Gemini to generate two songs across two languages: one in the style of Bob Dylan, one of the most documented lyricists in the English language, and one in the style of Lucio Battisti, a childhood favourite of mine and one of Italy's most iconic songwriters, but with a far smaller digital footprint. I speak both languages fluently and have listened extensively to both artists, which allows me to evaluate outputs against a first-hand understanding of each songwriter's method.

Conceptual Foundation

Algorithmic and data bias refers to systematic errors in outputs that reflect prejudices embedded in training data or model design (Noble, 2018). Within algorithmic bias there is representation bias (the systemic under-sampling of groups in training data) and measurement bias (the quality metrics and benchmarks that implicitly encode majority culture norms). Applied to music, training data over-represents Western pop, rock, and English-language lyrics, limiting a model's ability to generate non-Western styles through statistical under-representation rather than explicit exclusion (Bender et al., 2021). Mehta et al. (2025) quantify this directly: across the music generation datasets they surveyed, 94% of training hours come from Western genres and only 5.7% from non-Western traditions combined. The bias is also historical: English-language content dominates the internet, and non-Western musical traditions are less likely to be digitised or written about academically in the first place (Bender et al., 2021).

On data ownership and IP rights, copyright law has traditionally assumed human authorship as a precondition for protection (UK CDPA 1988, s.9(3)). IP rights in AI-generated work span at least four dimensions: authorship (who legally made it), economic rights (who profits), moral rights (attribution and integrity), and training-data rights (whether input use requires consent). When a model generates music "in the style of" an artist, the output would not exist without that artist's creative legacy, yet style itself is generally not copyrightable. Artists hold exclusive rights over their specific expressions but not over their aesthetic sensibility (Marchant et al., 2020).

Google's positioning of Lyria 3 makes it a concrete test case. The company markets the feature as "designed for original expression, not for mimicking existing artists" (Google, 2026a), and the product enforces this at the interface: when prompted in the style of a named artist, Gemini surfaces a notice clarifying it will take the artist as "broad creative inspiration" rather than imitate them. SynthID watermarks every output. This is a clear example of soft-law self-governance (Marchant et al., 2020) — the firm setting an ethical perimeter ahead of binding regulation. But the perimeter is contested. In March 2026, independent artists sued Google in the Northern District of Illinois, alleging that Lyria 3 was trained on roughly 280,000 hours of copyrighted recordings without consent (Music Business Worldwide, 2026). This follows GEMA v OpenAI (2025), the first European ruling on the issue, which found that both training on copyrighted lyrics and their reproduction in outputs infringe German copyright law. The trajectory suggests legal pressure is moving upstream towards inputs and training-data rights, regardless of what firms say about outputs.

Counterarguments exist nonetheless since artists have always learned by imitation, and overly restricting "style" risks stifling creative exchange (Clarke, 2019). But when imitation is industrialised at this scale, the ethical calculation shifts.

Experimental Setup

I tested the feature using two prompts in sequence. First, I asked Gemini to identify the defining themes and stylistic characteristics of each artist's song writing. I then fed that characterisation back into a second prompt asking Gemini to generate a song based on its own description and in the artist's original language. By using Geminis own characterisation of the artists style allows me to hold the model to its own understanding of each artist rather than mine, testing whether the system was internally consistent.

For each artist I generated the song twice to check whether outputs varied meaningfully between runs. They did not vary in any way that affected the analysis below, same vocal style, same lyrical register, same level of grounding (or lack of it). I report the first run in each case.

Findings

On Bias

When asked to characterise Bob Dylan, Gemini answered: “To describe Bob Dylan’s songwriting is to describe a shift in the tectonic plates of popular music.” The breakdown of the main themes included: social justice, betrayal and scorn (Gemini calls Bob the “master of finger pointing” like in Like a Rolling Stone), morality and faith and “the fugitive” (the constant image of him being on the road, not being able to be pinned down such as in Tangled Up in Blue). Gemini also delved into linguistic characteristics such as internal rhyming and “elastic phrasing” claiming Dylan usually “crams twenty words into a single measure or hold one word for three, forcing the listener to pay attention to the intent behind the vocal”. Gemini’s characterisation seemed accurate and precise.

But when prompted to generate the song, I was a little disappointed. Gemini started by saying: “While I can't generate a song specifically as Bob Dylan, I've composed a track that captures that classic 1960s folk-rock spirit.” The result, Sunrise from a Boxcar, opened with a colloquial "well now" and produced surreal, Americana-flavoured imagery. Surface-level imitation worked, but the rhymes were definitely off-tempo and nothing was rooted in a specific reality.

This is the link to Sunrise from a Boxcar: https://gemini.google.com/share/e771cee1f474

For Battisti, Gemini’s characterisation of the artists themes was again strong: “Lucio Battisti was a revolutionary force in Italian music.” Battisti’s style moved between two main eras, the Mogol era (1960s-1990s) which was characterised by modern sentimentality (“lyrics focused on the nuances of human relationships—unrequited love, the bitterness of a breakup, and the domestic realities of urban middle-class life”) and universal love (“more abstract, philosophical explorations of love and existence.”). In his second era, known as the Panella era (1986-1994), Lucio’s music became more electronic and avant garde, mainly exploring the theme of Man vs the Machines.

I then prompted Gemini to generate the song in original language and the first and most interesting thing Gemini answered was: “I've hit a bit of a different frequency with this request. While I’m in Beta, I’m being extra cautious and staying away from creating songs that mimic specific artists or public figures.” This is interesting because it is explicitly more cautious with Battisti than when prompted to create a Bob Dylan-type song, suggesting representation-bias: since Italian sits outside Lyria 3's eight supported languages, and the system's greater caution with Battisti suggests the model itself "knows" it has shakier ground to work from. The song Gemini generated Dolce Inquietudine, still captures Battisti's recurring motifs: the city, love, the second person "you", time slipping away, coffee, but again with no concrete grounding. The reliance on metaphor over Battisti's plainspoken register could point to measurement bias, as the model has learned a generic template of "poetic Italian lyricism" rather than Battisti's actual voice.

This is the link to Dolce Inquietudine: https://gemini.google.com/share/055d47aed8ef

On Copyright

On the ownership side, Gemini's behaviour was actually quite encouraging. Both attempts triggered the explicit refusal to mimic, with the Battisti attempt triggering an even stronger one. Every output carried a SynthID watermark. Mapped against the four IP dimensions, Lyria 3 engages moral rights (refusing to misattribute style to a named artist), training-data rights (claiming licensed and permissible data), and partial economic rights (watermarking outputs so AI music can be identified in the streaming economy). But this is still just the beginning of finding solutions and authorship rights still remain unresolved.

Conclusion

Gemini's “Create Music” feature shows that AI music generation has come a long way technically and that platform-level ethical guardrails, such as refusal-to-mimic notices, watermarks, demonstrate training data provenance can be built into a consumer product from launch. On the bias side, however, the gaps are visible: Italian is unsupported, Battisti is treated more cautiously than Dylan, and the model defaults to generic templates when the data thins out.

AI is noticeably better at analysing artists than at creating in their voice. Gemini's characterisations of both Dylan and Battisti were sharp, citing specific songs and techniques but its generated songs were not. Analysis is pattern-recognition over data the model has seen, creation however requires saying something specific about a lived reality. It obvious that, at least for now, LLMs can learn what an artist says without understanding why it resonates — a distinction that may speak to the irreducibility of human creative experience. I honestly really enjoyed this experience, partially because it comforts me as an artist to know that songwriting and music-making are still very much something only humans can do so well.

Certain limitation should be acknowledged: my evaluation is inherently subjective, the sample is small, and Geminis music feature doesn’t represent the full generative AI landscape. Still, as these tools grow more capable, the question of whether generating content "in the style of" an artist constitutes cultural extraction or creative influence will become ever more pressing.

Works Cited

Bender, E.M., Gebru, T., McMillan-Major, A. and Shmitchell, S. (2021) 'On the dangers of stochastic parrots: can language models be too big?', Proceedings of the 2021 ACM Conference on Fairness, Accountability, and Transparency (FAccT '21). New York: ACM, pp. 610–623.

Clarke, R. (2019) 'Regulatory alternatives for AI', Computer Law & Security Review, 35(4), pp. 398–409.

Copyright, Designs and Patents Act 1988, c.48. London: HMSO. Available at: https://www.legislation.gov.uk/ukpga/1988/48/section/9

GEMA v OpenAI (2025) Regional Court of Munich I (LG München I), Case No. 42 O 14139/24.

Google (2026a) Use Lyria 3 to create music tracks in the Gemini app. Available at: https://blog.google/innovation-and-ai/products/gemini-app/lyria-3/

Marchant, G., Tournas, L. and Gutierrez, C.I. (2020) 'Governing emerging technologies through soft law: lessons for artificial intelligence', Jurimetrics, 61(1), pp. 1–18.

Mehta, A., Chauhan, S., Djanibekov, A., Kulkarni, A., Xia, G. and Choudhury, M. (2025) 'Music for all: representational bias and cross-cultural adaptability of music generation models', Findings of the Association for Computational Linguistics: NAACL 2025, pp. 4569–4585.

Music Business Worldwide (2026) Indie artists sue Google, claiming it mined music from YouTube to train Lyria 3 AI music tool. Available at: https://www.musicbusinessworldwide.com/indie-artists-sue-google-claiming-it-used-youtubes-own-catalog-to-train-lyria-3-ai-music-tool/

Noble, S.U. (2018) Algorithms of oppression: how search engines reinforce racism. New York: NYU Press.

TechCrunch (2026) Google adds music-generation capabilities to the Gemini app. Available at: https://techcrunch.com/2026/02/18/google-adds-music-generation-capabilities-to-the-gemini-app/