test augmented reality

[nay-seven]

if you have a web cam and a printer :

a cool experience to do

[runagate]

that's amazingly beautiful

I've always wanted to incorporate this kind of physical modelling into an AR HID:
http://www.box2d.org

[multiphone]

Hello,

Very Nice,

But after a long conversation with Senso about physics modeling and box2D in Usine, the real question is :

"Wath kind of value are realy useful for composition in physics modeling ?"

I'ts a long way to make something intresting with that and to integrate in Usine i'ts another story...

But for me it's still a good idea.

Best

PS: For memory; http://www.sensomusic.com/forums/viewtopic.php?id=1816

[23fx23]

woa nice. interesting thread, iteresting questions.
That make me wanna find the time to make back some experiments via connecting usine to 3DSmax and communicate with ThinkingParticles, an awesome nodal patch oriented soft, somehow really similar to usine logical. It includes all classic forces/physics simulators, that can be realtime or precomputed, 2D or 3D, and should theorically be able to refeedback some var results to usine!)..sadly 3ds for now only catch midi, and couldn't find away to emmit back, script could write some txt, but a bit complex for now, even if i suceffully tested with midi, usine sending some enevoppe follows that made my animations react smoothy linked to sound, tryppy. i sooo wish an osc in_out or any way to make it and usine comunicate!!... ... you probably know that autechre video that must have used similar links:
http://vimeo.com/26692

it's rule event based, so can make some very modular simulations, via modules, wires, groups and forces, or script, just exactly like in usine!. as an exemple:
If SPACE SHIP GETS NEAR (VALUE) A ROCK
CHECK IF
BIG (VALUE) ROCK THEN AVOID IT (move around)
SMALL (VALUE) ROCK IGNORE AND COLLIDE
END// and the ship will make is own life in the asteroid field, without entering a single animation key.

now imagine if osc, a rock could become a note trigering chords on colissions, disepearing to frequency fragments or whatever, the "instrument" visual and sound scenario are totally linked together to the triggered code.
create some auto generating notes crowd rules,dominos effect,elasctic ropes fx control, complex collisions making sparky harmonics,cymbals ect.. nice electronica clips ne? make some bowling as a break, could in a way play every trippy audiovisual theatre scenario live. (let's add a wiimote as hid in on top of that )
if some wanna look the impressive endless possibilities; (while most exemples are realistic physics modeling, imagine this can do anything from semifluid/metaballs simulations like here, but also be turned in completly different directions to do total abstract 2D irrealistic ,make some squares growing, change colors, trig some videos, animations, swich some lights on, in fact quasi anything man can imagine, as modular in visual as what usine is to sound:

TP-3
Arf why no osc in 3DS...can't implement it martignasse ?;) think it's C sdk hehe

. The main problem of physics in our cases imo, is that the algorith/pgm must be straight linked to the user interact/visual feedback window that uses it, ie if a ball and elastic, man would want to visualise the elastic waves. I don't imagine and could understand usine could integrate internally such thing, at least what i imagine personnally is too exigent in cpu/graphic code and programming with is a heavy dedicaced job , and would need openGL/directX. of course TP is an extrme dynamic engine, in most musical cases we don't need such power. but in this ex, i m quite sure some usine XY balls could be send via osc to the dedicaced physic engine that would reproduce the contening box, lauch simulation after checking velocity in and balls would behave like in real, colliding with all imaginable possible scenarios/parameters settingz,, then sending back osc, so we would see the result in usine. (most basic exemple, there are millions more interesting thingz to do.).
cause if just bouncing balls,
man doesn't want to bother in such long and complex setup to play it's instrument and don't care seeing realisic 3d simulation,.Aren't there other more accessible 2Dphysics softs/libs that could communicate with usine ie via midi/ Osc , take some ins, compute physics, and return results?or that box2d soft seems a really good free challenger, do you think will it be a really hard to integrate via sdk?
...or maybe 2) finally the physics we are after, in fact linked to usine items, ( XY,steps,faders) are not that complex to script, (im thinking about lemur ones). while orbit and elastic must be more complex, the most used, inertia is quite simple i guess, as i could patch it to try with few items and delta. elastic is based on more complex inertia with a midle, bouncing needs angles computations ect..but a step by step a per item physics script or patch bank start could be really nice..

.. or maybe 3)in fact we just want senso script to not start random but go in the direction we send ball:lol: hehe. joking
ps sorry for too long post but im really totally "a donf" on the ideas here.

[multiphone]

Hello 23fx thanks for your response,
Ok, one by one, steps by steps… I try to give you back
First: For the moment I think that 3D S Max is certainly a good idea for graphics stuff via OSC, but my problem is: “I’m totally lost in this software” and I need to upgrade myself for making some interested things in this kind of environment.
Thinking particles are a real nice approach but I’ts another world for me too.
I think I don’t like very much to use a lot of software on stage in my own risk.
Second: The implementation of the box2D library is in progress for Max/MSP and the works of Mathieu Chamagne is the only multitouch application of them with audio.
http://www.youtube.com/watch?v=Os2Wzqm0dI0
They are graphical very nice and talented and create a sort of visual captivation for the public but I ask my self to wath can I do musically with this kind of mechanical interactions. Unfortunately I can test it easily with Usine and the “rebond friction” script are a little limited but very nice for a beginning. I’m searching a way…
Third : I know the works of Autechre and I like that is this style the works of CHDH
http://www.youtube.com/watch?v=N2V1FXHukgU
Or Servovalve : http://www.servovalve.org/
Are another exemple of the continuation of this kind of this thinking.
The new way come from multitouch and the interaction the human gesture for the musical composition and graphical interaction. A sort of “augmented cinema”
So…
Finally the questions is:
“Where is the good aproch?”
For me the only one is the sound and the real pleasure of create music and considers the images as a new possible instrument to create music and not vice versa.
All my apologies for this long post a little off topics and in my special English from the Est.
All of my best

[23fx23]

nice stuff multiphone,
oh yup definitely some searching, and experiment is the best to do. in my case i more think about 3Ds/tp for bringing some audio visualization/graphic experience, but i totally agree with you it 's some too complex and not intutive solutions to really think using them as live instruments, better focus on what you/we really want to control with physics and music.

while we can visualize physics, best would be to really feel it in the instrument, somehow have a "physic" feedback, im thinking as exemple at joysticks that have force feedback, if can interact on feedback force via usine, some sensitive experiences could be interesting maybe. that's what truely miss me more in electronical controllers, especially touch screen, the lack of physic feedback one traditionnal musician really feel..

[nay-seven]

thanks for the links Multiphone !
interesting links

here a first quick attempt in VVV, support OSC and could be a good candidate


btw, 23fx, there's a plugin in vst who can receive osc , maybe could interest you :
OSCGlue

[23fx23]

woa cool nay, didn't knew that vvvv seems very interesting! have to test that.

[multiphone]

Hello 23 and Seven,

VVV ? First time i see this software... a good cadidate for Alpha testing something,

Thanks Nay ! You are allready in top of the news !!

@23Fx: Your perfectctly right !

One of the best way to interact with 3D graphic stuff with music is to have some "physics" retroaction. The first time i have seen this kind of reactions is in "Phase project" with IRCAM.

http://recherche.ircam.fr/equipes/analy ... index.html

So a lot of works for the Usine comunity.

Best

[nay-seven]

thanks hervé , the phase project with the " rejouer la musique " video is interesting
about VVV , it's not really a news...2 years ago , I've try some first video midi controller ( so easy with capture module now.), and links was posted here..
check here

[23fx23]

yes very interesting, the haption devices looks really interesting, also i guess they are sadly still expensive solutions, maybe to democratise in future... i really wonder i one could manage (apple?) bring some cool tactile feedback to touchscreens one day...

nay, yup vvvv seems really cool for visual 2D stuff, a bit of my fault we have two topics in one, visual and physics(wich anyway are somehow linked) but maybe vvvv can handle some physic algorithms?. another free interesting solution for visual only is elektronika modular vj soft, that recently went totally free,
it's reason like modular, quite easy to use and really, really powerful. v2 was using midi, don't know if they implemented osc in 2.5.
sadly the official site seems down now, hope it's just temp maintenance...i guess we can find elektronika 2.5 somewhere else over the web...
http://www.aestesis.org/

about video capture there are propbably nice things to try out actually for free with usine module with pysics, ie bending a spring let us feel force feedback while video track data, (but think wiimote with IR led will always be more accurate in such tracking purposes, and higher speed/rez. there are really tons of things to experiment with ir leds as well, ie some transparent bouncing ball could have a ir led in center, or fix one in an elastic rope center, one led ring to our finger manipulting any "real" object than can have interesting physics feeling properties...
that virtually only cost a battery, ir led and wiimote, and a bit of imagination..
supose you saw it but look how reactive is the first fader moving, it's quasi instantaneous, so could ie manipulate ie a vertical elastic rope to feel some physics..
http://www.youtube.com/watch?v=uPWKYfTAM7s

[Clearscreen]

I've played around a lot with vvvv in the past, but struggled to get anywhere good - it can be a bit tricky to get your head around, but coming from that to Usine made Usine a breeze!

haven't tried elektronika for about 15 years, but d'ling now to have another look. Video jockey gold (Vjo Gold) recently went freeware as well - might be worth a look. As far as slightly less daunting stuff than vvvv there's also Isadora, but I never found it to be that stable...

I've used Mxwendler on and off for the last few years, and for straightforward VJ'ing it's pretty good - a little bit quirky, but good. It'll use midi, OSC and DMX as well. Hendrik's generally open to people beta testing his software for free if you get in touch with him, that's how I've had it.

[waolelaid]

Hi 23fx I saw the video but how do you connect the wiimote and the computer which devices do you need
I understand the IR pen but don't know with the wiimote if you can explain
thank you
regards

[23fx23]

hi waolelaid!
the wiimote is a bluetooth device, so the computer need eitheir build in bluetooth or bluetooth usb stick.
then if searching bluetooth devices in control panel and launching detect while pressing booth 1 and 2 butts of wiimote, it should recognise it as "hid input device". then can either use it directly in usine with wiimote module, or control windows mouse if using
jonny lee whiteboard app (google it, easy too find), that's what i used in the video.
the wiimote stands behind me looking at the screen, and the ir led is on finger, basically it tracks the ir dots so better have less angle and 2x screen width distance min, whiteboard then ask for a calibration with 4 corner cross then it's done. led can also be a pen or whatever...hope it helps!

[iococoi]

hi guys!

might be something usable:

http://musart.dist.unige.it/EyesWeb/EywResearch.html

or

http://www.derivative.ca/

overall tool shack:
http://www.audiovisualizers.com/toolshak/toolshak.html



maybe we could cook something up.


best wishes,
io

[23fx23]

nice stuff as well iococoi, this thread is full of interesting links!

[waolelaid]

thank you 23fx I understand now how it works I saw the johnny lee 's video but
I didn't understand all he explains because I didn't understand the role of the wiimote
the question of "hid input device" what's the difference with the web cam
they aren't identified as the same way by the computer I'm going to search
what is hid input device I believe there is a thread about this on the forum
but this thread is really interesting lot of things to learn
thank again
regards and good evening

[23fx23]

hey wolelaid,

in fact the wiimote is kind of wireless webcam, but classics webcam have poor rez (generally 640x480), they have
also cheap composent wich make shutter is low speed, max 30fps and there is a huge amount of motion blur, ie when moving fast, also they are color with high compression leading to artefacts.
on the contrary wiimote is a high quality 1024x768 BandW camera opertaring at 100Hz/50fps, and the big difference is that it as a UV/ir filter in front of it (the purple/black plastic in front) so in fact it doesn't see normal lights and colors, but only IR light. that makes that normally wiimote sees all black, if, and only if there is a concentrated IR light, ie a IRled, it will see a bright small spot in the midlle of the black. whereas a webcam would see normally, so lot of colors and lum variation hard to track.(but if applying a ir filter like those found on remotes on webcam, and using irled, the webam, behave quite the same). But seeing
the wiimote have been thinked for that, games requires high/speed/precision. in a 40$ wiimote there is a high end camera, cpu, tracker, and accelerometers, plus it's wireless, so better use it!!!!
cause that really allows the tracker to be far more accurate than classic webcams, cause it's high contrasted and not altered by ambiant light, or other object moves, and even if moving very fast, doesn't suffer of motion blur, or at least 2x less. the tracker then send XY coordinates of seen dots to wii/computer via bluetooth, it's seen on computer as hid input device wich just mean human input device,not a webcam you can visualize as usual ( then it will tell you it's a game controller).(find your bluetooth icon, it varies depanding manufacturers, but usually in control panel, and lauch detect devices holding1and2 on wiimote, then choose 'pair'). once computer and wiimote have been paired, usine or whiteboard can receive the infos from wiimote.

how it works: imagine what sees the wiimote, it sees like predator lol, it sees a bright dot going from ie left to right, so it sends XY infos, extracted from build in high precision tracker, that are jusy applied to mouse, jonny lee calibration tool is use cause ideally what sees the webcam should be the whole screen and only, so that we get max rez and accurate positionning, wich is not the case generally, so it maps datas to the calibration that was done to compensate angle/scale via clicquing on edges cross when launching wiimote whiteboard.
hope it helps good luck in experimentation!

[waolelaid]

Hi 23fx

Congratulations for this precise explanation
I understand now the difference and how it works
and going to experiment as soon as possible
thank you once again
best regards