ARTS 4880: Interdisciplinary Research Seminar - Music & AI

Arts Department, Rensselaer Polytechnic Institute

Tuesday/Friday, 12:00-1:50am, SAGE 2510

Instructor: Rob Hamilton, West Hall 305/6

e: hamilr4 [at] rpi [dot] edu

w: http://homepages.rpi.edu/~hamilr4/arts4880

Office Hours: TBD

COURSE SYLLABUS - Spring, 2024

PREREQUISITE

ARTS 2380 or ARTS 2020, or permission of instructor

COURSE DESCRIPTION

Advances in machine learning and artificially-intelligent systems are currently being applied to creative tasks such as musical composition and performance. This course is an advanced seminar focusing on the current state of AI research and application in musical domains. Topics to be discussed include the ethics of AI's use in music, the way(s) that trained systems are being applied towards the creation and performance of music and the hands-on application of AI/ML toolsets and frameworks for musical expression.

CATALOG DESCRIPTION

This is a course introducing music majors to advanced research topics of the Rensselaer music faculty. Each semester a member of the music faculty will focus the seminar on a research topic or paradigm related to their own body of artistic and technological research. Sample topics might include Spatial music and sound, New Instrument Design, Network Music, Music Information Retrieval, Ethnomusicology, Sonification Art and Science, Music and Logic, Spectralism and Beyond, Music Herstory (feminist music composition), Experimental music and sound history. Through hands-on creative research, students will explore questions of both musical and technological significance while engaging that same topic through their own hands-on creative practice.

LEARNING OUTCOMES

    Students who successfully complete this course will demonstrate...
  1. an understanding and appreciation of the current statue of AI/ML systems for musical creation
  2. basic technical facility in the application of current tools and frameworks for AI/ML assisted composition, source separation, classification, sound synthesis and post production.
  3. creativity and resourcefulness through the creation of musical AI/ML systems and creation/composition of their own sonic projects

EVALUATION

Evaluation is based on the following:

CLASS PARTICIPATION

You will be required to present some of your assignments to the class, to show your work within the software environment you used to create it, and to engage the class in discussion of your work. When you are not presenting your own work, you need to be attentive to whoever is presenting, and to engage them in discussion of their work. Failure to participate in class will lower your grade.

ATTENDANCE

You must attend class to succeed in this course.

  1. Since much of the class is focused on listening to and discussing work in class, attendance is mandatory.
  2. ** More then two unexcused absences will affect your grade, detracting 1/2 grade each additional 2 unexcused absences. **
  3. Late arrivals are very disruptive - continued late arrival will affect your grade.
  4. It should go without saying but no use of mobile devices or personal computers during class time (except for as required by the coursework itself) is acceptable. Continued violations will be treated as an unexcused absence.

STATEMENT REGARDING ACADEMIC INTEGRITY

Collaboration between students in this course is strongly encouraged. Likewise, students are encouraged—indeed, to some extent required—to exchange ideas, opinions and information . You are also encouraged to help each other in the lab and with performance, production, and presentation of composition projects.

Plagiarism of any kind is in direct violation of University policy on Academic Dishonesty as defined in the Rensselaer Handbook, and penalties for plagiarism can be severe. In this class you will be expected to attribute due credit to the originator of any ideas, words, sounds, or music which you incorporate substantially into your own work. This applies particularly to citation of sources for sonic "samples" included in your compositions.

The use of automated technical aids (e.g. ChatBots, AI code plugins) is not strictly forbidden but if used MUST be documented in great detail and discussed with the Professor prior to use in a graded project/assignment.

Submission of any assignment that is in violation of this policy may result in a grade of F for the assignment in question. Violation of this policy will be reported, as defined in the Rensselaer Handbook

DISABILITY SERVICES FOR STUDENTS

Students requiring assistance are encouraged to contact Disability Services: http://doso.rpi.edu/dss to discuss any special accommodations or needs for this course.

OFFICE HOURS

Office Hours for Spring, 2024 will be TBD

TEXTBOOKS / REPOSITORIES / FOUNDATIONAL MATERIAL


COURSE SCHEDULE

The proposed course topics and schedule will be as follows (take note of project due dates!). Based on class progress and interests, this schedule is subject to change. Special topics, guest lectures, supplemental reading, listening and additional assignments to be announced.

Week 1:
Friday, 1/12

Algorithmic Composition: David Cope and EMI
http://artsites.ucsc.edu/faculty/cope/

Voice before, and in the age of AI

THE VODER @ Bell Labs
Long (audio + images): https://www.youtube.com/watch?v=5hyI_dM5cGo
Short (video): https://www.youtube.com/watch?v=0rAyrmm7vv0

Who owns a song?
Fogerty vs. Fantasy
Blurred Lines: Thicke/Pharrell vs. Estate of Marvin Gaye

Who owns a voice?

Midler vs Ford: https://en.wikipedia.org/wiki/Midler_v._Ford_Motor_Co.
Ford commercial: https://www.youtube.com/watch?v=hxShNrpdVRs
Midler version: https://www.youtube.com/watch?v=QGpwgSo3THE

Waits vs Frito-Lay: http://law2.umkc.edu/faculty/projects/ftrials/communications/waits.html
Commercial: https://www.youtube.com/watch?v=bNIJWCNEzTw

Week 2:
Friday, 1/19

Due: Tone Transfer Sketch

Simple interfacers/instruments for "public" use: Ocarina by Smule: https://www.youtube.com/watch?v=ZoYz4uM9RgY
Feedback loop - Alvin Lucier's "I Am Sitting In A Room": https://www.youtube.com/watch?v=fAxHlLK3Oyk

Week 3:
Tuesday, 1/23

Machine Learning and Music
George Lewis: Too Many Notes: Complexity and Culture in Voyager (2000)
George Lewis demonstrates Voyager: https://www.youtube.com/watch?v=ncy4_FHX3Jc
George Lewis "Why Do We Want Our Computers To Improvise" (Talk): https://www.youtube.com/watch?v=wDP8FsjyCaA

Friday, 1/26

Fun with ChucK: https://chuck.stanford.edu/
* Joystick Chanting: Joy of Chant
* ChucK AI: ChAI, AI Tools
* Ge Wang (2/16): CCRMA Homepage

Chuck/Markov example: smrtMarkov.ck
More on Markov Processes: Simoni: Algorithmic Composition Using Probabilistic Methods
Lisp Markov Example: markov.lisp

Rebecca Fiebrink's WEKINATOR
http://www.wekinator.org/

For class on Tuesday (1/30): Select one AIMC paper and present it to the class
* AIMC Archives: https://aimusiccreativity.org/conferences/

Week 4
Friday, 2/2

NO CLASS

Reembodied Sound Symposium @ EMPAC: https://empac.rpi.edu/events/2024/reembodied-sound/schedule-keynote

Week 5
Tuesday, 2/6

Introduction to ChucK: Syntax, Direction and Scope
- MiniAudicle: ChucK IDE
- Web Chuck IDE: https://chuck.stanford.edu/ide/


Hidden Markov Models (the silent killers): https://towardsdatascience.com/markov-and-hidden-markov-model-3eec42298d75
Jurafsky & Martin: https://web.stanford.edu/~jurafsky/slp3/
- HMM Appendix: https://web.stanford.edu/~jurafsky/slp3/A.pdf

ChAI HMM Example: https://chuck.cs.princeton.edu/doc/examples/ai/hmm/hmm-load.ck

Friday, 2/9

Introduction to ChucK continued: time => now, loops, shreduling, hid devices, envelopes

Week 6
Tuesday, 2/13

Introduction to ChucK continued continued

Friday, 2/16

Ge Wang Visit: CCRMA Homepage

ChucK Community Discord: https://discord.gg/RGkZracy
* ChucK AI: ChAI * Reference: AI Tools


Week 7
Tuesday, 2/20

NO CLASS (Presidant's Day Observed)

Friday, 2/23

NO CLASS (? TBD)

Week 8
Tuesday, 2/27



Friday, 3/1



Week 9
Tuesday, 3/5

NO CLASS (SPRING BREAK)

Friday, 3/8

NO CLASS (SPRING BREAK)

Week 10
Tuesday, 3/12



Friday, 3/15



Week 11
Tuesday, 3/19



Friday, 3/22

Guest Speaker: Anna-Kaisa Kaila, PhD Candidate, Creative AI Researcher
KTH Royal Institute of Technology, Media Technology and Interaction Design
http://www.kth.se
https://www.creative-ai-project.se/


Week 12
Tuesday, 3/26



Friday, 3/29



Week 13
Tuesday, 4/2



Friday, 4/5



Week 14
Tuesday, 4/9



Friday, 4/12



Week 15
Tuesday, 4/16



Friday, 4/19



Week 16
Tuesday, 4/23

FINAL CLASS