Who wants MIDI in the Flash Player?
The challenge:
The Flash Player has evolved through the ages to provide the most needed functionality. Through each version there have always remained a few common goals. What I have found is that:
Flash is small — from the player itself to the swf file format to the assets it is optimized to load, focus has been placed on small file sizes (this of course is not as apparent in many websites that are heavy in multimedia)
Flash supports standards — the player supports many web and multimedia formats standard in the industry, such as jpg, mp3 and xml
Flash is interactive — the players greatest strength is the dynamic behavior through ActionScript to allow user interactivity
MIDI, a music standard format that most computers support today, fits all of these categories (like a glove). In fact there’s an opensource project being developed to allow MIDI through Flash, though it requires an additional download and install to the user apart from the Flash Player itself (seen at osflash.org)
I’d like to take a poll. Do you think MIDI should be included in the Flash Player? Why or why not? I want both votes and opinions as I’ll organize the results and send them off to “Adobe, formerly known as Macromedia”. Please respond with some sort of opinion whether it’s pro or con. I’ll list the pros/cons I can think of below (you don’t have to read the rest of this email if you already have your opinion).
A little more on MIDI:
MIDI is a standard music format (some will argue that it’s the only standard) that represents pitches and instruments to be played as a song. It’s extremely small, being the vector of music, and has to be interpreted by a users soundcard. Almost all computers these days support standard MIDI, though it sounds synthesized (especially on the voice and string instruments). Some soundcards or additional software transform the common MIDI into amazing orchestrations, but most users don’t have this advanced playback.MIDI pros:
can be generated dynamically and played through a sequencer to allow complete on-the-fly customization of sound.
very small in filesize
supported by almost all soundcards
numerous applications for the creation of MIDI songs (many are free)
it’s a standard that has been around for a long time (so there is a lot of support for it)
a small implementation (wouldn’t increase the Flash Player size by more than 50K)MIDI cons:
most people will have a more synthesized sound
user experience isn’t guarenteed to be consistant (for those with higher quality soundcards)
as with all advancements, could make it really easy for developers to have annoying sounds playing on their sites
In short, if the Flash Player had a midi sequencer built in it would allow developers to create lightweight interactive music applications, such as this sheet music rendering application or music creation applications. It could also allow users to experience a website that contained sound effects or decent background music at very little bandwidth cost. Macromedia has always been good about listening to the developer community and will surely make efforts to build the features we need, provided we tell them.
In sending this challenge to the Flashcoders mailing list I received a great response. Here I hope to summarize their contributions acurately to their credit. (also see full conversation)
MIDI Sound
Making MIDI sound available in the Flash Player would give “major advantages over WAV and MP3 [with] tiny file size and the potential for allowing Flash to create music dynamically, even interactively.” It allows for flexibility in sound that can be readily available to the developer and “opens some new possibilities for using flash and actionscript in a creative way.” Regarding the industry “making MIDI available through Flash will open another market for Adobe. Right now graphic designers and programmers are using Flash, so having musicians on board would definately make Flash a universal tool.” It appears that there are many developers already creating solutions less desirable to fill a need. If Macromedia were to fill that need I am confident there would be a movement toward dynamic audio applications greater than that of design. “I believe adding MIDI to the Flash Player could have similar catalytic creative effect to when Atari first built a MIDI port into their computers — opening the floodgates to some of the best original sequencer software like Dr Ts. Commodore Amiga did the same thing when they included Video in gamut. Revolutionary in its day and still some great ideas from that machine have not reappeared.”
Once again, the drawback is “lots of bad tunes,” a “lack of quality control — somewhat akin to using system fonts with which there’s no guarantee of consistent text layout, only with MIDI the synthesizers in most sound cards are cheesy to say the least.” It is mentioned that “general MIDI banks embedded in the player” would bloat the size, but others say a MIDI solution would actually be very light weight and add little to the player. There is also a concern about device and driver compatability but it seems each OS abstracts those layers in their own way. A significant project already underway in cross platform MIDI/Flash solutions is Alexis Isaac’s Flash MIDI.
Devices
A broader use of MIDI is its compatability with many external devices. It could come as a simple “api for accessing midi devices on the user system, much like accessing a webcam, where you query for which devices are present, then get a handle to a device and start manipulating it.” “If there were low-level interfaces for it…you could attach devices that use midi…and use the midi commands to control Flash, and vice versa…. I don’t have any idea what kind of Flash content could put such controls to use, but the best way to find out would be to make it possible, then sit back and see what people do.” “In addition to keyboards, drum boxes, synths, FX devices and PCs, there is a valuable world of MIDI control surfaces…. MIDI in the Flash Player opens up a fascinating new world of using Flash to drive an exciting range of external media hardware…. Given the skill of programmers on this list I would not be surprized at what emerges, especially when used WITH…other MIDI-friendly hardware and software.”
There is the possibility of communicating with devices through Socket connections, or having an application-specific communication layer as an additional download/install. But “MIDI bandwidth demands can be very misleading. So routing via XML sockets…is great for some applications and hopeless real-time performance-wise for others.” It is possible, “but it would just be nice to minimise the number of pieces in the puzzle.”
Sound Buffer
Access to MIDI controls would be a great step for the Flash Player, but an api to control a sound buffer could prove the most powerful solution: “a more open, low level approach to sound in the player upon which MIDI and other implementations could be developed.” “A memory area that allows you to create sound waves. It is a sound buffer (which can be…inside a flash sandbox) which is no different than a buffer that holds strings or bitmaps…. One can have lots of sound buffers…that [hold] data in 16bit (or whatever) wave amplitudes.” “With a simple sound.setBuffer(binaryData) you could do a huge amount of interested things…. Sound synthesis, generation.” Using mathematics to create sounds instead of recordings “could be done with a minimal impact on filesize” in the deliverable. With “Sound capture…you could capture audio from mic and allow the user to edit it and process it. You could build annotation tools, voice messaging, musical applications etc…. Also if you can get a handle on the audio stream before it hits the audio device you could have live control over streamed audio, tone controls, reverb, delays, echo cancellation, noise reduction, etc.”
Some are concerned that by allowing access to an audio buffer there could be security vulnerabilities such as buffer overflows. These concerns are valid and can be read in full context, but are also preventable.
_______________________________________________
There has been such a large contribution just from the Flashcoders list that I’ve decided to post this publicly to allow everyone to add their voice. Please contribute to the ideas and names listed here as a petition for more audio capabilities for the Flash player.
The player will have to do a better job of allowing the user the option of silencing flash. I can see it being a huge frustration for a user who is listening to his iTunes and having it clash with a site that doesn’t give him the immediate option of turning off the flash-generated garble. I think with the advent of flash MIDI you’d see a lot more use of sound in flash content. Not to mention that “Kill the monkey”-type ads will be even more annoying, if that is possible.
The Pro: user experience
The Con: user experience
Yes, MIDI is really nice to communicate with the other devices and/or application. This way you could make a Flash tool which can communicate with a Gypsy suit etc. That would be awesome. Yes, binary sockets could be something nice, maybe it’s possible to write some sort of service/daemon that redirects incoming MIDI stuff into a Flash readable format i.e. binary sockets or xml. Hmm, maybe a nice school project
I was thinking the same thing, Josh, about the right-click mute functionality.
I say go for the MIDI! I think I’m with you guys, I love those monkey ads, it is so fun to shoot them and punch out Osamma Bin Laden! That is what flash is for!
Seriously, I do think we should include the MIDI. It is all about power to the people. If they do something dumb with it, then let them, but just allow the sound to be turned off, so that it doesn’t take away people’s God-given free agency.
John Dowdell from Adobe responded on the list.
He asks, “which specific MIDI (or MIDI-like) features & implementations are most desired; and (b) how much [would you] be willing to pay for this in player size?”
see the full conversation here
I believe a General MIDI soundset and a MIDI playback routine is already included in both the Mac and PC operating systems since at least 5 years. (On the Mac it is an integral part of QuickTime, eg.) Without it, one wouldn’t be able to play .mid files. (You can test for its presence by opening http mysite.verizon.net/mrdelurk/BluesInF.mid or http mysite.verizon.net/mrdelurk/A1Demo_snippet.mid – if your computer plays music from any of these files, you have both the soundset and the MIDI playback engine installed.)
So, as far as Flash goes, having a copy of the General MIDI soundset might be less necessary than having the ability to handle the MIDI data going to it – to record that MIDI data, and to allow editing of it, becoming a brain for the playback engine. (In MIDI lingo, to act as an entry level “MIDI sequencer”)
On one level up, if you don’t like the sound quality of the General MIDI preset sounds of your computer, I can see two choices. The first is to piggyback on QuickTime again and explore its “Atomic Instruments” capability. The other is to expand Flash Player’s sound playback routine, instead. One can add AIFF or WAV sounds to Flash files for simple playback already, now it would be a matter of upgrading its playback ability with polyphonic pitch shifting. (In MIDI lingo, enabling it to act as an entry level “software playback sampler”)
All in all, one could upgrade Flash Player in many ways, the change you need the least is to add *another* General MIDI soundset to your computer when it already has it
Only my 2 cents, of course
Previous poster again. I found a way how to get Flash and MIDI playback work together. You can even start / stop MIDI playback from Flash. To see a working example (created without any special Flash extensions), visit http home.hawaii.rr.com/reginafierro/mainNS.html. The MIDI sequence will start playing automatically. You can stop / restart it by clicking on the Music On / Music Off links inside the full browser Flash movies.
The site is an fantasy art gallery BTW, with some nudity within good taste. If you need a step by step description how I welded MIDI and Flash together leave a message here and I’ll type it up the next time I hop by.
This method doesn’t allow for dynamically generated midi. It seems to play a midi file fine on my Firefox browser, though I wonder how well it is supported universally.
The method involves javascript creating an embed tag (so it is HTML, not Flash, that plays the MIDI)
for anyone interested, this is the code
The above ‘midi’ hack, is not really what most people around here are looking for.
So I fiddled around a bit, with the suggestion that was brought up earlier — dynamically generate a swf into a ByteArray, then use Loader.loadBytes to load it in as if it were a normal swf. I was able to get something working, while it’s not perfect, you are able to create different sounds. With some more work, it could be extended, and the quality cleaned up a bit.
Example on my site, requires 8.5 beta 2 to listen:
http blog.davr.org/2006/04/21/dynamic-sound-in-85/
Nice David! I think this sort of thing has a lot of potential. I’d like to make some time to start mixing and patching together MP3 instruments.
No new update on FlashPlayer support for MIDI. It looks like we’re going to have to wait for more official Adobe announcements. Don’t count on it for Flash 9, anyway.
I believe the pros far out-weigh the cons. One of the things I’d like to see the most is a speed control, and perhaps even a pitch control for that matter. Imagine being able to change the speed, or pitch, of any midi file in real-time.
Also, it would inspire people to make higher quality samples for the midi instruments.
Do IT!
Who wants MIDI in the Flash Player?
I WANT!!!
What ever became of this? Does the current version of Flash support MIDI? Does Adobe have any plans to support it in Flash? I’ll happily sign a petition or vote Yes if it’ll help!
Unfortunately the word out is that MIDI will not likely be implemented in the Flash Player at all. The reasoning behind this is that, where the Flash Player stands for ubiquity and common experience, MIDI is inconsistent between platform and device.
Because Adobe is moving toward dynamic sound generation, it is more likely that MIDI will become available through an ActionScript-built sampler. Adobe looks to the community to provide these “edge case” solutions.
It would be fantastic midi in flash
gerrix
Hi i have found a few external workaround/plugin/server/applet (none of them perfect) to use midi in flash.
http://board.flashkit.com/board/showthread.php?t=789516
and still no midi in flash10
MIDI IN FLASH NOW !!!!
I like the layout of your blog and I’m going to do the same thing for mine. Do you have any tips? Please PM ME on yahoo @ AmandaLovesYou702