One of the most exciting areas of music technology development is happening in the realm of gaming and interactive virtual space. Music and Sound Design play crucial roles in the design of gaming environments, narratives and flow. And as designers create ever more innovative game experiences featuring rich graphics, fast multiplayer networking and next-generation controllers, new techniques for creating immersive music and sound for games to complement and showcase these advances are not only possible but necessary.
This Studio class will explore cutting edge techniques for building interactive sound and music systems for games and 2D/3D rendered environments. To better understand the link between virtual space and sound, students will learn the basics of designing sound and composing music for interactive game spaces by designing and implementing rich musical games within the Unreal gaming engine. Coursework will require the ability and desire to code game logic and design game environments. Techniques for integrating sound and music within games including game-centric middleware tools like FMOD and WWISE, interactive sound synthesis and computer networking using Open Sound Control will all be explored.
Working in teams or on their own, students will design their own music-rich game experience, compose music, design sound material, and implement their own playable musical game experiences.
Students will explore the artistic role of music and sound in gaming by building their own interactive sound and music-rich games and 2D/3D rendered environments. Within the context of their own creative game projects, students will learn the basics of designing sound and composing music for interactive game spaces. Using workflow programming languages and software tools, students will program basic gaming interactions, link them to interactive audio software, and create a musical gaming experience.
Students who successfully complete this course will demonstrate...
- an understanding and appreciation of game design and musical creation through an awareness
of the many disciplines underlying the field including: software design and programming, interaction design, listening skills, musical theory,
musical acoustics, digital audio theory, and digital signal processing.
- basic technical facility in the areas of game development, audio recording, editing, sound synthesis, software development and post production.
- creativity and resourcefulness through the creation of musical game environments and composition of your own sonic projects
Evaluation is based on the following:
- Projects and compositions (75%)
- - Assignment #1: Sound Effects and Field Recording (5%)
- - Assignment #2: Designing "Sound Rooms" (5%)
- - Assignment #3: Composing within New Atlantis (15%)
- - Assignment #4: Sound-Only Game (20%)
- - Final Project: Musical Game (30%)
- Attendance/class participation (25%)
You will be required to present some of your assignments to the class, to show your work within the software environment you used to create it, and to engage the class in discussion of your work. When you are not presenting your own work, you need to be attentive to whoever is presenting, and to engage them in discussion of their work. Failure to participate in class will lower your grade.
You must attend class to succeed in this course.
- Since much of the class is focused on listening to and discussing work in class, attendance is mandatory.
- ** More then two unexcused absences will affect your grade, detracting 1/2 grade each additional 2 unexcused absences. **
- Absences can only be excused by a letter from a medical doctor or from the Dean of Students' office.
- Late arrivals are very disruptive in Studio 110 - continued late arrival will affect your grade.
STATEMENT REGARDING ACADEMIC INTEGRITY
Collaboration between students in this course is strongly encouraged. Likewise, students are encouraged—indeed, to some extent required—to exchange ideas, opinions and information . You are also encouraged to help each other in the lab and with performance, production, and presentation of composition projects.
Plagiarism of any kind is in direct violation of University policy on Academic Dishonesty as defined in the Rensselaer Handbook, and penalties for plagiarism can be severe. In this class you will be expected to attribute due credit to the originator of any ideas, words, sounds, or music which you incorporate substantially into your own work. This applies particularly to citation of sources for sonic "samples" included in your compositions.
Submission of any assignment that is in violation of this policy may result in a grade of F for the assignment in question. Violation of this policy will be reported, as defined in the Rensselaer Handbook
DISABILITY SERVICES FOR STUDENTS
Students requiring assistance are encouraged to contact Disability Services: http://doso.rpi.edu/dss to discuss any special accommodations or needs for this course.
The proposed course topics and schedule will be as follows (take note of project due dates!). Based on class progress and interests, this schedule is subject to change. Special topics, guest lectures, supplemental reading, listening and additional assignments to be announced.
Introduction: Designing Musical Games :: Gaming Musical Design
- The role of sound and music in games; filmic influences; spectra of interactivity
Introduction to Pure Data
: history, language basics, download
Open Sound Control
: namespaces, OSCsharp
C# classes, integration with Unity
PD + Unity: Roll-A-Ball
OSC demo project (download
Pure Data/OSC output to Unity: Roll-A-Ball pt. II
Basics of sound in Unity: AudioClip
, Audio Source
: editing and processing sound files (download
Manual Link: Audio
Recording simple sound effects: here
Recorders: Tascam DR-40
, Marantz w/external mic @HASS Equipment room
Sound Effects Exercise #1:
Sounds and Spaces
- Using field recorders from the equipment room, create a set of sound assets for either the Roll-a-Ball project or for the Spaceship project
- Edit your recordings using Audacity.
- Add your sounds back into the Unity project.
Review and Discussion for Assignment #2 Proposals
How to design an abstract musical game-space for New Atlantis
NEW ATLANTIS WORKSHOP
NEW ATLANTIS WORKSHOP
NO CLASS (MONDAY SCHEDULE POST-HOLIDAY)
Pure Data: Sampling, audio file playback, buffers, timing structures
Generative Music Systems: Leonard Paul's music from "Sim-Cell": youtube
Pure Data: step-sequencers, graph-on parent,
Running Pure-data patches on mobile devices
Guest Lecture: Ezra Teboul (celldrone
ALT-CTL: Alternate controllers for musical games (midi keyboards, wiimote w/osculator
, Leap Motion
Unity Teapot: Alvin Lucier's "Nothing is Real" for game engine and Leap Motion
Pure-Data/Audio game Q/A session
Pure-Data/Audio game Hack-day
Assignment #3: In-class presentations
Final Project Proposals
Final Project Hack-A-Thon
THANKSGIVING HOLIDAY - NO CLASS
Final Project Hack-A-Thon
Final Project Proposals - Final Check-ins, Progress Reports
Final Project Presentation Play-testing session
Final Project Presentations and wrap-up