My Digital Journey Prt2
..moving on to 2010 and beyond, a painful road a head. Wild west of scanning, a no-mans land of information and wiring perils a plenty.
Scanning Strikes Back!
Scanning Strikes Back!
This was the year of growth. Slow but steady. I was starting to phase out my DI kit from 2009-10 and starting to experiment with other custom solutions. I invested in some Artec L’s (which was a very bad move) and had plans to invest in a Mephisto structured light system (which subsequently I didn’t do and bless the lucky stars I didn’t!)
I also begin to learn Softimage this year thanks to help from the very talented Paul Smith (aka Pooby from BustyKelp) then I purchased Arnold thanks to Paul egging me on. Arnold was incredible, the first rendering application I’d ever used that included true volumetric subsurface scattering. This enabled me to experiment with realistic skin shading and lighting to produce some very cool results. Before Luc Begin and then Adam Lewis set the trend in the industry for using VraySSS2 (back in 2009 and 2010) which now all studios seem to favor, even Weta tried to buy Luc’s setup, it was that good!)
But in my mind Arnold was light years a head of everything else and still is. It’s just stuck in Maya and Softimage (now also available for Lightwave)
This back-end training and research enabled me to release a high resolution scan to the community.
This was a HUGE success, being downloaded over 100,000 times in over 3 years. Used in 100’s of videos and technical papers, also featured in many power point presentations and simulation demos in the computer graphics industry for events like Siggraph. No one had openly done this before, not without including various contracts or legal papers for usage and rights. A few people abused the moral rights of the scan, even though it was covered under the creative commons license.
One example was my head digitally cut in two, with mosquito’s flying out! for a cover of a malaria magazine in Africa. As IF that’s the right moral usage! Anyway, the guy took it down (liberty!!)
There’s a great thread on CGFeedback (fantastic site!) that shows how people used the head scan / sculpt. This post has amassed nearly 150,000 views over a few years. Very cool to see people sharing ideas and talking about CG methods openly!
It’s very strange seeing your digital head all over the cg industry. At this time I was also really trying to push being open about ideas, and sharing techniques. Much like Paul Smith does on Vimeo. It’s hard to take this approach, shying a way from NDA’s and being secretive. I think being open about techniques helps others grows and ideas flourish. For example sharing that 3D scan set for free, brought in a huge amount of work for IR and also helped raise IR’s profile, whilst allowing many studios, researchers and individuals data to test with and learn from.
Using Softimage and Arnold. Trying to push the boundaries of digital-double realism.
This was the summer of frustration with DSLR’s and learning about Photogrammetry. I wasted countless weekends, countless Friday nights standing in my underpants trying to scan myself. To no avail. Hours, Days, Months of my wasted life on a seemingly never ending challenge. This was the time I realized when there was really no one to ask, no one to email for guidance or advice. I just had to blindly believe that what I was doing WOULD work. I knew it would. I BELIEVED.
One Friday night, when I knew I should’ve been out boogying at the local ‘discotheque’ (they call ‘em that these days right? Instead I was sat in my tiny studio, the floor and my feet covered in wires and tripods. I mean 100’s and 100’s of wires, cameras every where. Thinking “what the F*CK am I doing with my life”
…apparently according to a small voice in my head, “patience is a virtue”.
Early full body photogrammetry results using a DI kit. (AWFUL!!)
These were captured with 6 cameras, off grid, off the DI system, custom wired. 3x stereo pairs that each had to be calibrated with 15 sets of images for each pair taking about 20 minutes each to take. Nearly an hour just to calibrate. Damn what a pain in the arse. As you can see the results were less than desirable, even with 6x 12MP cameras. I knew I was on to something but the DI kit was lacking. I needed something else. I was close ..but so far.
Then I saw a Youtube video from XYZ-RGB.
Hot dawg! 16x Camera array! 16x. 16x Cameras! Incredible! Maybe some day.
These guys were pioneering this field, way before anyone else. All current 3D photogrammetry systems, I believe, owe homage to these guys, as they taught me allot and I’m certain allot of people have learned the techniques they use, from IR. They originated (feel free to correct me otherwise) with Helmut Kungl, who was a real inspiration. Soon to release Scanner-Killer in 2011.
Towards the end of 2010, I continued to learn more about 3D rendering in Softimage and how to manipulate 3D scans to workable subdivision models.
Early Softimage Arnold and 3DSMax Vray renders, using Kel Solar’s awesome sIBL GUI tools. Big thanks to Luc taking the time to teach me Vray and Kel and Paul for showing me Softimage, all over MSN and Skype.
Cutting edge 360 face capture! early 2010. 10x Canon 450D. Custom system. The actual scans were very poor.
Some more commercial projects (low key), some articles and tutorials in 3DWorld magainze and a few encounters with people like DJ Goldie, Andky Serkis and Lionel Richie for some small private 3D scanning gigs. Great fun!
The dream still, was a fully automated, single shot, full-body scanning system!
Scanner Killer landed. After my visit to a 3D Scanning Conference in Lugano, I decided upon Scanner Killer instead of Mephisto (at this point Agisift Photoscan was just a twinkle in a 3D scanner’ers eye!)
This system was great because it was custom, open, out of the box software. You can calibrate your own cameras without special hardware limitations imposed through the DI kit. But again you HAD to calibrate and could only setup in stereo pairs. Pain. This was worse for calibration because you had to spend upto 30 minutes per-pair with a checkerboard calibration target. I even had to get a giant one printed! check it:
It was the only way to calibrate the system. Agisoft late 2011 would prove to be rather useful in this regard, with ZERO calibration required.
Still I had to earn my stripes. I had to do the ground work to fully understand the process. I envy people starting today, it’s so much easier to do. A walk in the park to build multi-camera systems than compared to pre-2010.
The Scanner Killer, single angle, stereo pair scans were good. Faster build time than DI but still noisy and not quite there. Some very obvious limitations. You could just imagine the issues with calibrating a 16x Camera array.. say clipping a tripod or moving a camera accidentally. Forget about it.
WOah! Space age!
Scanner-Killer also had issues with deep scanning, cut offs in the depth, limbs would vanish or cut out. A really bad ‘bug’ in the code, or just a limitation of the software. These parts had to be re-built / sculpted by hand. Not to mention distortion errors, warping, incorrect scaling etc. All due from the lenses and poor human calibration errors.
The body scans also had to be hand cut out of the background noise, a little bit like carving stone or archeology digging.
Early 4D Capture
This also freed me up to experiment with 4D techniques, capturing time with the DSLR’s although the heavy compression and low fps ruined the output. Still note worthy for the time and again, the first person to achieve it with Scanner-Killer then Agisoft.
1x Stereo Pair
2x Stereo Pair
(Later 2011, 4D Capture with Agisoft Photoscan)
12x Cameras with Agisoft later in 2012
What I liked to call the “U.K Noire” technique
Later in 2010, back again with Scanner-Killer I got to put the system into practice with my favorite model. Joceline.
Some of these images contain mild Nudity:
Using a 3 camera setup, 2 to capture noise from a single DLP projector and another camera to capture colour from a flash light delayed to over power the noise projection. Complex to arrange, had to be reliable. We were also using a motorized Ortery turntable. Taking multi-angle shots. Prone to subject movement. Here was the first welded 360 degree output, after many hours of hand stitching.
Useless! but kind of respectable, still, a long way to go. This technique required too much post processing. At this point I was still debating what to do. Was this the right decision buying a Pro Scanner-Killer license. Should I sell my Artec L’s which proved to be no more use than paper weights.
This was also the first time I had figured out how to use ReMesher (before Dynamesh) and how to UVMaster unwrap the UV’s on the scan, to project and bake details. This was doing it with over 8 different angled scans, all cut, trimmed and welded. Damn hard, precision work and prone to artistic interpretation and error. Not desirable.
Artec L’s failing miserable (way before Hao Li’s Non-Rigid, we also met this same year!)
Then I learnt about Agisoft Photoscan thanks to Samuel Poirier and Merry Hodgkinson (we would soon collaborate allot! clever guy ol’Merry!) from the CGFeedback forums (with their awesome test scans using a single DSLR)
Merry and I toying with the old DI kit and Merry’s multi-lighting creation! one of many future meetings to come.
So I began to test further…
2011 (mid way)
Agisoft Photoscan had arrived, and was kicking ass. 100% comparable with Disney’s reconstruction code from Zurich, yet not grounded to stereo pairs. This was a true Volumetric Reconstruction approach. Meaning in theory, the more cameras you threw at the software, the more kick-ass it was. INCREDIBLE! and my saving grace. This software alone made all the last few years of battling worthwhile. There was light at the end at the overly’ dramatic tunnel!
I began throwing data at the software and the results were wonderful. Something new, something fresh. Breaking barriers and time. Very fast. I then upgraded to Canon 550D’s first 3, then 6, then 8 and then I bought 18x of them to test with. Pretty much unheard of in the UK.I always seemed to be about 1-2 years a head of the competition, due to my tenacity and lack of social life. “I have a competition in me”
This new setup was all crudely wired, with USB2 and 2.5mm stereo trigger leads all with batteries! But I got the thing to work and synced thanks to XYZ-RGB’s clever light syncing technique (no noise projection required, pure PASSIVE!)
Much higher fidelity, still using a turntable and cutting / stitching data. But I was free from calibration and the results were far sharper and much higher resolution due to having more cameras being combined and producing a volumetric scan. Combined with multiple megapixels of colour information!
This was when the larger companies started to pay an interest. My first international cross border deal with Microsoft.
ooooouuush! the first company to create a 360 degree (stitched) full body scan, in 2011 with Agisoft Photoscan.
*Note, you will notice I use. “note”, “first-time”, “world first”, ” first in the world”, ” first in the country”, “first!” ALLOT. As they say, “history is always written by the victors” Please quote me if I’m wrong.
Then a client called in the US representing Nike called:
“Lee, can you build a single shot full body capture system with 60x cameras?”
unheard of.. but
“Yeah, no problem”
“Great because we need to scan the fastest woman on the Planet!”