3D Virtual MikuMiku Dancing

hatsune-miku-virtual-dancing

3D graphics combined with positional tracking and camera recorded video combine with Hatsune Miku in this striking video:



In case the exposition doesn’t interest you, after the dancing there is an additional interesting video at 3:15.

More from the same animator:



This line of technology is very promising indeed….


    Post Comment »
    65 Comments
    Sort by: Date | Score
    Comment by Kyougamikutan
    23:20 12/04/2009 # ! Neutral (0)

    I want the 3d file with RIG... D:
    all i have is the model and im not good at rigging so i can't animate smoothly.. =\

    Comment by Anonymous
    12:01 20/09/2011 # ! Neutral (0)

    I am an excellent 3D studio max animator and could easily rig her with physique and add jiggle-bones to the hair for a perfect animation model

    source for model?

    Avatar of ichimokuren
    Comment by Ren Ichimoku
    23:08 12/04/2009 # ! Neutral (0)

    so scary When Its alive

    BTw Its Cute ^_^

    Comment by Coolness of cool
    04:59 13/04/2009 # ! Neutral (0)

    In the first video, does anyone know the name of the song. It sounds so familiar. probably nintendo like. Anyone help plz.Onegai shimasu!

    Comment by Vetus
    08:53 13/04/2009 # ! Neutral (0)

    Say, what's the title of this video in order to search for it on YouTube?

    Avatar of Firaku
    Comment by firaku
    Comment by Anonymous
    21:41 12/04/2009 # ! Neutral (0)

    anyone else laugh their ass off at the end of the first vid, i thought pirates of the carribean was taking over hatsune miku, possibly the only thing scarier than a space pirate ninja, is a space pirate ninja vocaloid.

    Comment by lin
    21:31 12/04/2009 # ! Neutral (0)

    http://www.nicovideo.jp/watch/sm2570873
    ?

    Avatar of lerrymaru
    Comment by maru
    15:31 12/04/2009 # ! Neutral (0)

    Yarrr looks like our 2D girls will be materialized soon enough! WIN

    Comment by Algester
    18:33 12/04/2009 # ! Neutral (0)

    mmm since this is also partly my course field at least now i have some ideas about how to use camera tracking

    Avatar of metatron
    Comment by metatron
    21:30 12/04/2009 # ! Neutral (0)

    very impressive...

    the more you know~

    Comment by Anonymous
    19:05 12/04/2009 # ! Neutral (0)

    I for one, welcome our new Saatchi overlords.

    Comment by Ooohh
    10:30 28/08/2009 # ! Neutral (0)

    Okay we need to stop making computer animation and robots now.

    Comment by Anonymous
    02:38 28/08/2009 # ! Neutral (0)

    My spaceship! They stole it!

    Comment by Anonymous
    15:36 17/01/2010 # ! Neutral (0)

    damn that ship looked so awesome

    Avatar of Hentaikid
    Comment by Hentaikid
    11:37 18/01/2010 # ! Neutral (0)

    Ah syntheyes, I really should buy a copy.

    LunarSD> You might want to check out lesswrong.com they talk a lot about possible AI

    Comment by another Anon
    12:16 31/05/2009 # ! Neutral (0)

    japan is ready for North Korean lol

    Comment by Anonymous
    20:31 23/04/2009 # ! Neutral (0)

    also see it
    http://www.nicovideo.jp/watch/sm4230739

    Comment by Anonymous
    03:48 12/04/2009 # ! Neutral (0)

    AR IS AWESOME!!

    Avatar of dmil666
    Comment by dmil666
    03:43 12/04/2009 # ! Neutral (0)

    That's pretty slick. That shrine/building clip was especially pretty.

    Avatar of bomblol
    Comment by PoisonedV
    03:41 12/04/2009 # ! Neutral (0)

    Ahaha oh my god the second video is awesome

    Avatar of Obsidian
    Comment by Obsidian
    03:49 12/04/2009 # ! Neutral (0)

    As expected; It's growing rapidly.

    Avatar of Fatty
    Comment by Fatty
    03:52 12/04/2009 # ! Neutral (0)

    Hair physics too surrealistic for me

    Comment by Xero
    03:54 12/04/2009 # ! Neutral (0)

    Aww wish there was more. Next we need a 3D Miku vs Rin fighting scene. See, once and for all, who is the superior vocalist.

    Comment by Zig
    04:30 12/04/2009 # ! Neutral (0)

    or MikuxRin 3D yuri

    Comment by Otakumanichiwa
    03:38 12/04/2009 # ! Neutral (0)

    hmm I want that animator....NOW!

    Avatar of LunarSD
    Comment by LunarSD
    03:18 12/04/2009 # ! Neutral (0)

    A vision of the anime chick robot apocalypse to come?

    Avatar of Artefact
    Comment by Artefact

    It looks as if augmented and virtual reality companions will precede robots by a decade or two...

    Avatar of LunarSD
    Comment by LunarSD
    03:35 12/04/2009 # ! Neutral (0)

    You better believe they will. I haven't been spending the last 3 years architecturing digital neural and synaptic ecosystems in C++ and writing a CUDA based physics engine for nothing. Anime archetypes will birth into reality, if not by my tested hands, then by those who will come after.

    Comment by AT
    17:57 13/04/2009 # ! Neutral (0)

    Sounds cool, and I'd like to know more. Any chance you'd care to share (wisdom, code, other things of note)?

    (I'm a programmer. I plan on learning OpenCV sooner or later)

    Avatar of LunarSD
    Comment by LunarSD
    02:04 14/04/2009 # ! Neutral (0)

    http://www.nvidia.com/object/cuda_education.html

    Modeling brain dynamics, physics, lighting, vertices, textures, etc. all at the same time is insanely computationally expensive. So just a regular Quad core computer wont do it on its own.

    Apart from making networked code to run on multiple computers (which would work too, but be more of a hassle IMO), programming on the new generation of "general purpose" graphics cards is the way to go.

    Over a hundred ~1.5 Ghz processors on a modern el cheapo graphics card (several hundred, & SLI scalable, in the really good ones), and only FOUR ~3 Ghz processors on an expensive Quad core.

    And if you write your code to be doing different computation sets on the CPU while running your GPU kernels, it stands to reason you can be crunching Physics on the GPU while computing the domino-like (waaaaay oversimplifying here) forward and backward flow of data between synapses on the CPU.

    Then switch them up depending on which segments of code are best handled massively parallel (send as a CUDA-scripted kernal to the GPU to handle) and which ones do fine in serial (crunch it, still threaded but less dependent of parallelism, in C# or VB.net or whatever).

    ATI has their own language too, so if you don't like nVidia feel free to learn the competitor's stuff.

    But if you follow the above link, nVidia basically lets you download an entire semester's worth of course material for free, so that's the path I took. They obviously want people to use their technology so more customers are forced to buy their cards in the future. That's fine IMO, it's damn fine technology, worthy of surviving this recession (fingers crossed >_< ).

    -----

    If you want to model brain dynamics and physics, those are the 2 most parallelizable computational problems I can think of. Everything is happening concurrently, and in doing so effects everything else concurrently, iterate to next frame, repeat.

    AI's need an interactive environment on par with what we humans are forced to battle against every instant of our lives. Invariances occur in the Physics of our environment, we learn patterns of them, and receive/contemplate feedback along our merry way. Again, oversimplifying here. I've got systems for dreaming, goals, rationalizing, you name it. And NOT in silly logic trees like video game "AI"'s. I call it a neural/synaptic ecosystem for a reason.

    I'd love to go on and on about how far I've already managed to get with my own Homebrewed code (no Newtonian physics here, I model down to chemical bonds, albeit Vertex-sized ones...), but that's impossible inside this tiny, tiny box.

    For the best intro you can expect to get to AI that's not stupid logic trees and meaningless heuristics, I suggest this book: http://www.amazon.com/Intelligence-Jeff-Hawkins/dp/0805078533/ref=sr_1_1?ie=UTF8&s=books&qid=1239635204&sr=8-1

    Do not fall into the pitfall of wasting time on anything the Academic community published on AI as a field. Studying up on Probability, however, is a good idea. Neurons and Synapses essentially form a structure built on evolving invariances, probabilities and mutations. Mostly in their connecting plasticity, but also in their protein wall gate behaviors and 3D nearness to other clusters they can just "spray" into. All perfectly abstractable into computer code. Reading up on Neuroscience is pretty much required. You don't have to get too deep into it though, there's plenty of simple to grasp flash animations of cell signaling dynamics all over the 'net.

    Yes the old Academic Neural Nets and probability Matrix algorithms can do simple Voice Recognition and stupid Robot motions, find the "beat" in a song and bob along mindlessly to it, and so forth.

    But frankly no one's even bothered putting serious effort into Sentience, or as they're calling it these days "general purpose AI".

    Notice how it's "general purpose" GPU (the GPGPU), and "general purpose" AI? Narrow application vs. Generalization. One gives you a fat paycheck if you majored in it, the other gives you endless possibilities if you're brave enough to sacrifice the time/life needed to harness the beast. Obviously, I live a meek, poor life. But I have never, EVER had the opportunity to get bored with it :D

    The AI community mostly just cares about publishing white papers about simple narrow areas they can apply kids toy neural networks to that have been around since the 70's so they can collect their paychecks. Nothing wrong with it, but where's the progress?

    This mindset, this fear that they wont be taken seriously if they work towards general purpose AI, has done NOTHING but hold the AI community back in its own stone age.

    AI wise, I also read a little Ben Goertzel (some good ideas of emotion, moods and so forth), but wasted too much time on AI history. So empty of anything remotely clever enough to become sentient.

    The only practical successes I got came out of sitting in my chair at work, pretending to be working, but really thinking of my own adaptations of highly evolvable "ecosystems" of neurons and synapses, writing notes to remember and later evolve on top of old ideas. Working through ideas on how neurons and synapses interact to encode/decode data between specialized clusters, the ability to call up memories of percept configurations and imagine sensory forms in the "mind's eye", the way data flows from the Cortical layers to the Hippocampus and back again, what it's doing along the way, why certain optical illusions become perceived the way they do, what dynamics work to cause Ego formation, what mechanically is occurring to maintain the subconscious, how Heterarchies work versus how Hierarchies work, etc etc etc.

    Even if you drop any big ambitions you may have along the way, it's an excellent learning experience to better yourself with. Philosophically as well as intellectually.

    But I've got it all covered. Don't waste your life on something like this if you're not sociologically insane, such as, for example, I am. I don't mind giving up everything that's construable as regular healthy social living as long I've still got my dreams to chase ahead of me.

    I've lived a good enough life even before I first heard the word "anime" (was 19 at the time), so putting everything I can muster towards this work, and occasionally taking breaks for anime and so forth, is all I really need any more.

    That's not enough for an awful lot of people to be satisfied with.

    You fellow otakus just enjoy your life paths, 2D and 3D alike, and be patient. 2D is going to evolve, you have my word on that. But word or not human curiosity will still herd it in that direction.

    If someone (not necessarily me) does make a human-level AI, it's got to grow up developmentally before anything especially productive can come of the technology.

    Granted when it's older he/she can be cloned indefinitely, but there has to be at least one raising period, preferably while feeding them tons and tons of anime ^_^ (and a good education too, and some moral fiber, obviously)

    Bah sorry for the tl;dr.

    Trust in the passion of the otaku spirit. We're not exactly sane enough to end up NOT inventing something that could change the world, for better or for worse.

    For what it's worth, I've taken especial pains in contemplating how one might design neural clustering architectures that are incapable of falling into illogical "bigoted" ways of thinking as their thoughts abstract up the Cortical pathways, and the ability to have greater executive control over emotional cycles.

    So if I'm not a complete idiot, they shouldn't end up gullible fish, and should be equipped to pick up on dangerous fanaticism and general irrationally-based varieties of discrimination well before they settle into the subconscious. In other words, they'll hopefully be better off that we are in how broad of a perspective they get to see the world with.

    Sorry I withhold on a greater depth of specifics, it would just take too many days of writing at this point. Better it stays in code where it has immediate practical application.

    Alright I'm cutting myself off before this gets too much longer.

    Thanks for showing an interest though. Fruit will be borne as soon as I am able.

    Probably starting as a Youtube intro to how the GUI works while you're arranging the neurons and synapses, a little discussion on engineering/design concepts, what you can't neglect versus what you could (for instance, you couldn't possibly get away with making an intelligent AI if it didn't have touch/pain sensors and a challenging Physical environment to have to overcome; it's too fundamental to positive/negative feedback foundations and basic linguistic analogies, it wouldn't have the capacity to learn to communicate significantly, among other deficits and pitfalls, such as not being able to empathize with the pain of others and becoming psychopathic).

    Maybe in 2 or 3 more years at this point, who could say. It's more work than it sounds.

    Avatar of LunarSD
    Comment by LunarSD
    02:08 14/04/2009 # ! Neutral (0)

    I can't believe I just wrote that much serious minded text on a MikuMiku dancing video blog post -_-;

    Maybe this way it will only benefit the right sort of people...

    Comment by Anonymous
    20:32 28/08/2009 # ! Neutral (0)

    zzzzzzzzzzzzzzzz oh im in sankaku right? zzzzzzzzzzzz

    Avatar of Kioswan
    Comment by Kioswan
    16:47 24/08/2009 # ! Neutral (0)

    It's always good to see someone who works passionately at something for the better of everyone.

    Comment by AT
    18:11 14/04/2009 # ! Neutral (0)

    I am in awe. Yeah, I've already heard of CUDA, but I never looked for educational material. Thanks.

    Watching anime, reading sci-fi, and in particular exposing myself to Asimov's works really got me interested in computer intelligence. I recall being disappointed by my AI class that I took two years ago - it was essentially state-space search and statistical machine learning, with no real cognitive implications whatsoever.

    Thank you for your extensive post - it was an interesting read. Is there any place online (web or IRC) that you hang out, where I can follow your work? =) When I have time, I hope to be able to follow along, myself - and perhaps, given time, to help.

    Comment by Himura
    14:03 14/04/2009 # ! Neutral (0)

    damn long text is long.

    Avatar of LunarSD
    Comment by LunarSD
    00:16 15/04/2009 # ! Neutral (0)

    Thanks for listening/reading. I used to bounce these sort of thoughts off on my brother (4.0 family genius; I'm the other side of the proverbial weighing table who got sick of it all early on and dropped out to follow his dreams, a common sibling distinction?). Now that the project is nearing the real hatching period, it's a nice feeling getting some of it out there again.

    I'm pretty nomadic in cyberspace, though it would be nice to find somewhere with a similar minded crowd and share ideas. If you happen to know of such a place...

    Artefact, I imagine, would appreciate my not polluting his article comments with questionable ramblings any more than I already have :P

    I really should follow the open source Way, it's been closed circuit since day one. Maybe when things are refined enough to slap a Beta tag onto it.

    On the other hand, given how the maturity level for those given any degree of power (such as the responsibility of raising an AI) always seeks a local minima, I do hesitate at the thought of dumping this kind of code into the infinite possibilities of cyberspace.

    If I ever do get around to posting something on to Youtube, my name there is ThaddeusChristopher. Literally just joined. I can't say I respect the way they've been handling things of late, and the community drama looks asinine, but it's still the best place to give your ideas exposure.

    Feel free to hit me up in the comments (do they do PM?) if ever you're inclined.

    More mundanely, and anime related, I'll be uploading a few AMVs that I've been tinkered on to that account in a week or two. Test out the community. This avatar is a screenshot from one. I did it to learn video editing (who doesn't in this community?). That shot and most of the rest of the vid heavily uses bump mapping algorithms, lens filters, various FX chains, track overlay types, etc.; I like the painted look it ended up with. It's to Pink Floyd (Brain Damage & Eclipse) and the Theme from A Clockwork Orange.

    Avatar of LunarSD
    Comment by LunarSD
    01:37 15/04/2009 # ! Neutral (0)

    *tinkering... edit button DOKO DESU KA >:( ?!

    Comment by Anonymous
    11:21 12/04/2009 # ! Neutral (0)

    You, sir, are my hero. Thank you.

    Comment by chibiKEN
    03:17 12/04/2009 # ! Neutral (0)

    that looks super cute ...@_@

    Comment by Char
    03:24 12/04/2009 # ! Neutral (0)

    haha yea somewhat

    Comment by ⑨
    08:27 12/04/2009 # ! Neutral (0)

    みっくみくにされました

    Avatar of Gradius
    Comment by Gradius
    02:39 26/04/2010 # ! Neutral (0)

    SUGGEEEEEEE!!!!

    Avatar of Ari
    Comment by GourmetPrince
    03:30 12/04/2009 # ! Neutral (0)

    This is easily the scariest fucking thing I have ever seen in my life.

    Avatar of Ichiro Ino
    Comment by イチロ イノ Ichiro Ino
    03:27 12/04/2009 # ! Neutral (0)

    now the touch/texture is only missing in VR... If only it could be simulated...

    Comment by Anonymous
    14:52 12/04/2009 # ! Neutral (0)

    google haptic interfaces.

    Comment by Daisuke
    05:14 12/04/2009 # ! Neutral (0)

    fight rin FIGHT!!

    well anyways this was rather impressive!!
    And miku was so great!!

    Avatar of giascle
    Comment by iapfah
    06:42 12/04/2009 # ! Neutral (0)

    Impressive. Compositing videos takes a hell of a lot of work; especially for such a well-animated one. I've always wanted to try this but never had the tools outside of class.

    also lol: http://img10.imageshack.us/img10/1690/miosqueeze.jpg

    Comment by halotoo
    11:38 12/04/2009 # ! Neutral (0)

    medicore,seen better.

    Comment by Anonymous
    14:54 12/04/2009 # ! Neutral (0)

    [citation needed]

    Avatar of LunarSD
    Comment by LunarSD
    03:05 13/04/2009 # ! Neutral (0)

    We hope you have enjoyed this sample from our new model Anon version 2.0 #sankakuX-01.

    This latest upgrade removes previously overused phrases such as "sauce plz", "link?", "moar", and so forth in favor of a charming new set of witty ripostes and colloquialisms. A welcome change for all.




    Post Comment »

Popular

Recent News

Recent Galleries

Recent Comments