Animating a photo-real digital face - Paul Debevec
one<00:00:15.980> of<00:00:16.100> the<00:00:16.160> biggest<00:00:16.940> challenges<00:00:17.390> and
one of the biggest challenges and
one of the biggest challenges and
computer<00:00:17.930> graphics<00:00:18.350> has<00:00:18.500> been<00:00:18.680> able<00:00:18.800> to
computer graphics has been able to
computer graphics has been able to
create<00:00:19.310> a<00:00:19.700> photo<00:00:20.300> real<00:00:20.540> digital<00:00:20.960> human<00:00:21.680> face
create a photo real digital human face
create a photo real digital human face
and<00:00:21.980> one<00:00:22.369> of<00:00:22.460> the<00:00:22.520> reasons<00:00:22.820> that's<00:00:22.939> so
and one of the reasons that's so
and one of the reasons that's so
difficult<00:00:23.390> is<00:00:23.810> that<00:00:23.960> unlike<00:00:24.310> aliens<00:00:25.310> and
difficult is that unlike aliens and
difficult is that unlike aliens and
dinosaurs<00:00:25.670> we<00:00:26.390> look<00:00:26.570> at<00:00:26.720> human<00:00:27.050> faces
dinosaurs we look at human faces
dinosaurs we look at human faces
everyday<00:00:27.919> they're<00:00:28.250> very<00:00:28.430> important<00:00:28.820> to<00:00:28.880> how
everyday they're very important to how
everyday they're very important to how
we<00:00:29.000> communicate<00:00:29.570> with<00:00:29.720> each<00:00:29.779> other<00:00:30.050> and<00:00:30.590> as<00:00:30.680> a
we communicate with each other and as a
we communicate with each other and as a
result<00:00:31.130> we're<00:00:31.610> tuned<00:00:31.910> into<00:00:32.239> the<00:00:32.329> subtlest
result we're tuned into the subtlest
result we're tuned into the subtlest
things<00:00:33.260> that<00:00:33.410> could<00:00:33.500> possibly<00:00:33.920> be<00:00:34.010> wrong<00:00:34.309> with
things that could possibly be wrong with
things that could possibly be wrong with
the<00:00:34.610> computer<00:00:34.760> rendering<00:00:35.390> in<00:00:35.989> order<00:00:36.230> to
the computer rendering in order to
the computer rendering in order to
believe<00:00:36.470> whether<00:00:36.800> these<00:00:36.980> things<00:00:37.190> are
believe whether these things are
believe whether these things are
realistic<00:00:37.940> and<00:00:38.300> what<00:00:39.140> I'm<00:00:39.230> going<00:00:39.380> to<00:00:39.500> do<00:00:39.590> in
realistic and what I'm going to do in
realistic and what I'm going to do in
the<00:00:39.860> next<00:00:40.070> five<00:00:40.280> minutes<00:00:40.309> is<00:00:40.730> take<00:00:40.910> you
the next five minutes is take you
the next five minutes is take you
through<00:00:41.180> a<00:00:41.239> process<00:00:41.660> where<00:00:41.840> we<00:00:41.899> tried<00:00:42.350> to
through a process where we tried to
through a process where we tried to
create<00:00:42.680> a<00:00:43.040> reasonably<00:00:43.700> photorealistic
create a reasonably photorealistic
create a reasonably photorealistic
computer-generated<00:00:45.050> face<00:00:45.560> using<00:00:45.950> some
computer-generated face using some
computer-generated face using some
computer<00:00:46.430> graphics<00:00:46.790> technology<00:00:46.910> we've
computer graphics technology we've
computer graphics technology we've
developed<00:00:47.840> and<00:00:48.170> also<00:00:48.410> some<00:00:48.560> collaborators<00:00:49.010> at
developed and also some collaborators at
developed and also some collaborators at
a<00:00:49.340> company<00:00:49.700> called<00:00:49.730> image<00:00:50.180> metrics<00:00:50.570> we're
a company called image metrics we're
a company called image metrics we're
gonna<00:00:51.080> try<00:00:51.320> to<00:00:51.380> do<00:00:51.530> a<00:00:51.560> photo<00:00:51.830> real<00:00:52.040> face<00:00:52.310> of<00:00:52.580> an
gonna try to do a photo real face of an
gonna try to do a photo real face of an
actress<00:00:53.270> named<00:00:53.570> Emily<00:00:53.870> O'Brien<00:00:54.230> who's<00:00:54.500> right
actress named Emily O'Brien who's right
actress named Emily O'Brien who's right
there<00:00:54.860> and<00:00:55.190> that's<00:00:55.550> actually<00:00:55.760> a<00:00:55.940> completely
there and that's actually a completely
there and that's actually a completely
computer-generated<00:00:56.810> rendering<00:00:57.620> of<00:00:57.740> her<00:00:57.860> face
computer-generated rendering of her face
computer-generated rendering of her face
and<00:00:58.100> by<00:00:58.400> the<00:00:58.460> end<00:00:58.610> of<00:00:58.760> the<00:00:58.820> talk<00:00:59.000> we're<00:00:59.210> gonna
and by the end of the talk we're gonna
and by the end of the talk we're gonna
see<00:00:59.510> it<00:00:59.600> move<00:00:59.810> the<00:01:00.620> way<00:01:00.740> that<00:01:00.890> we<00:01:00.980> did<00:01:01.160> this<00:01:01.370> is
see it move the way that we did this is
see it move the way that we did this is
we<00:01:01.850> tried<00:01:02.060> to<00:01:02.270> start<00:01:02.750> with<00:01:02.900> Emily<00:01:03.380> herself<00:01:03.710> who
we tried to start with Emily herself who
we tried to start with Emily herself who
was<00:01:04.070> gracious<00:01:04.280> enough<00:01:04.549> to<00:01:04.699> come<00:01:04.850> to<00:01:05.239> our
was gracious enough to come to our
was gracious enough to come to our
laboratory<00:01:06.170> in<00:01:06.619> Marina<00:01:06.920> del<00:01:07.040> Rey<00:01:07.159> and<00:01:07.579> sit<00:01:08.210> for
laboratory in Marina del Rey and sit for
laboratory in Marina del Rey and sit for
a<00:01:08.479> session<00:01:08.689> in<00:01:08.930> Light<00:01:09.110> Stage<00:01:09.409> five<00:01:09.649> this<00:01:09.890> is<00:01:09.950> a
a session in Light Stage five this is a
a session in Light Stage five this is a
face<00:01:10.369> scanning<00:01:10.700> sphere<00:01:11.060> with<00:01:11.180> 156<00:01:12.079> white<00:01:12.409> LEDs
face scanning sphere with 156 white LEDs
face scanning sphere with 156 white LEDs
all<00:01:13.250> around<00:01:13.610> that<00:01:14.149> allow<00:01:14.360> us<00:01:14.390> to<00:01:14.540> photograph
all around that allow us to photograph
all around that allow us to photograph
her<00:01:15.320> in<00:01:15.680> a<00:01:15.920> series<00:01:16.250> of<00:01:16.310> very<00:01:16.580> controlled
her in a series of very controlled
her in a series of very controlled
illumination<00:01:17.570> conditions<00:01:18.020> and<00:01:18.260> the<00:01:18.590> lighting
illumination conditions and the lighting
illumination conditions and the lighting
that<00:01:18.950> we<00:01:19.070> use<00:01:19.220> these<00:01:19.460> days<00:01:19.520> looks<00:01:19.970> something
that we use these days looks something
that we use these days looks something
like<00:01:20.540> this<00:01:21.020> we<00:01:21.260> shoot<00:01:21.470> all<00:01:21.650> of<00:01:21.680> these
like this we shoot all of these
like this we shoot all of these
photographs<00:01:22.130> in<00:01:22.550> about<00:01:22.729> three<00:01:22.970> seconds
photographs in about three seconds
photographs in about three seconds
and<00:01:23.930> we<00:01:24.020> basically<00:01:24.140> capture<00:01:24.979> enough
and we basically capture enough
and we basically capture enough
information<00:01:25.880> with<00:01:26.420> video<00:01:26.990> projector
information with video projector
information with video projector
patterns<00:01:27.890> that<00:01:28.070> drape<00:01:28.340> over<00:01:28.670> the<00:01:28.700> contours<00:01:29.270> of
patterns that drape over the contours of
patterns that drape over the contours of
her<00:01:29.479> face<00:01:29.690> in<00:01:29.960> different<00:01:30.409> principal
her face in different principal
her face in different principal
directions<00:01:31.370> of<00:01:31.670> light<00:01:31.820> from<00:01:31.970> the<00:01:32.030> light<00:01:32.240> stage
directions of light from the light stage
directions of light from the light stage
to<00:01:33.020> figure<00:01:33.290> out<00:01:33.440> both<00:01:33.710> the<00:01:33.860> course<00:01:34.130> scale<00:01:34.400> and
to figure out both the course scale and
to figure out both the course scale and
the<00:01:34.700> fine<00:01:34.970> scale<00:01:35.299> detail<00:01:35.750> of<00:01:36.110> her<00:01:36.290> face<00:01:36.500> if<00:01:36.710> we
the fine scale detail of her face if we
the fine scale detail of her face if we
zoom<00:01:36.979> in<00:01:37.130> on<00:01:37.220> this<00:01:37.370> photograph<00:01:37.880> right<00:01:38.150> here
zoom in on this photograph right here
zoom in on this photograph right here
you<00:01:38.870> can<00:01:38.960> see<00:01:39.110> it's<00:01:39.229> a<00:01:39.320> really<00:01:39.590> nice
you can see it's a really nice
you can see it's a really nice
photograph<00:01:40.220> to<00:01:40.400> have<00:01:40.580> a<00:01:40.610> fur<00:01:40.909> because<00:01:41.210> she's
photograph to have a fur because she's
photograph to have a fur because she's
lit<00:01:41.720> from<00:01:42.049> absolutely<00:01:42.560> everywhere<00:01:42.979> at<00:01:43.100> the
lit from absolutely everywhere at the
lit from absolutely everywhere at the
same<00:01:43.430> time<00:01:43.760> to<00:01:44.240> get<00:01:44.360> a<00:01:44.390> nice<00:01:44.540> image<00:01:44.840> of<00:01:45.049> her
same time to get a nice image of her
same time to get a nice image of her
facial<00:01:45.350> texture<00:01:45.950> and<00:01:46.159> in<00:01:46.280> addition<00:01:46.729> we've
facial texture and in addition we've
facial texture and in addition we've
actually<00:01:47.420> used<00:01:47.630> polarisers<00:01:48.530> on<00:01:48.710> all<00:01:48.890> the
actually used polarisers on all the
actually used polarisers on all the
lights<00:01:49.190> just<00:01:49.580> like<00:01:49.640> polarized<00:01:50.450> sunglasses
lights just like polarized sunglasses
lights just like polarized sunglasses
can<00:01:51.229> block<00:01:51.470> the<00:01:51.710> glare<00:01:51.830> off<00:01:52.010> of<00:01:52.250> the<00:01:52.430> road
can block the glare off of the road
can block the glare off of the road
polarisers<00:01:53.840> can<00:01:54.020> block<00:01:54.229> the<00:01:54.439> shine<00:01:54.740> off<00:01:55.070> of
polarisers can block the shine off of
polarisers can block the shine off of
the<00:01:55.490> skin<00:01:55.729> so<00:01:55.939> we<00:01:56.030> don't<00:01:56.180> get<00:01:56.270> all<00:01:56.330> those
the skin so we don't get all those
the skin so we don't get all those
specular<00:01:57.110> reflections<00:01:57.680> to<00:01:57.860> take<00:01:58.040> this<00:01:58.189> map
specular reflections to take this map
specular reflections to take this map
now<00:01:59.030> if<00:01:59.090> we<00:01:59.180> turn<00:01:59.299> the<00:01:59.420> polarisers<00:01:59.960> around
now if we turn the polarisers around
now if we turn the polarisers around
just<00:02:00.530> a<00:02:00.619> little<00:02:00.830> bit<00:02:00.950> we<00:02:01.040> can<00:02:01.159> actually<00:02:01.280> bring
just a little bit we can actually bring
just a little bit we can actually bring
that<00:02:01.880> specular<00:02:02.570> reflection<00:02:03.049> of<00:02:03.140> the<00:02:03.229> skin
that specular reflection of the skin
that specular reflection of the skin
back<00:02:03.740> in<00:02:04.010> and<00:02:04.670> you<00:02:04.760> can<00:02:04.790> see<00:02:05.030> she<00:02:05.180> looks<00:02:05.390> kind
back in and you can see she looks kind
back in and you can see she looks kind
of<00:02:05.630> shiny<00:02:05.900> and<00:02:06.140> oily<00:02:06.290> at<00:02:06.560> this<00:02:06.740> point<00:02:07.040> and<00:02:07.280> if
of shiny and oily at this point and if
of shiny and oily at this point and if
you<00:02:07.759> take<00:02:07.939> the<00:02:08.060> difference<00:02:08.479> between<00:02:08.659> these
you take the difference between these
you take the difference between these
two<00:02:09.349> images<00:02:09.530> here<00:02:10.069> you<00:02:10.580> can<00:02:10.729> get<00:02:10.849> an<00:02:10.970> image<00:02:11.180> lit
two images here you can get an image lit
two images here you can get an image lit
from<00:02:11.959> the<00:02:12.049> entire<00:02:12.319> sphere<00:02:12.709> of<00:02:12.799> light<00:02:12.980> of<00:02:13.159> just
from the entire sphere of light of just
from the entire sphere of light of just
the<00:02:13.849> shine<00:02:14.150> off<00:02:14.450> of<00:02:14.629> Emily's<00:02:15.290> skin<00:02:15.560> I<00:02:15.590> don't
the shine off of Emily's skin I don't
the shine off of Emily's skin I don't
think<00:02:15.980> any<00:02:16.099> photograph<00:02:16.549> like<00:02:16.730> this<00:02:16.909> had<00:02:17.030> ever
think any photograph like this had ever
think any photograph like this had ever
been<00:02:17.330> taken<00:02:17.689> before<00:02:17.870> we've<00:02:18.200> done<00:02:18.379> this<00:02:18.560> and
been taken before we've done this and
been taken before we've done this and
this<00:02:20.120> is<00:02:20.329> very<00:02:20.389> important<00:02:21.019> light<00:02:21.200> to<00:02:21.379> capture
this is very important light to capture
this is very important light to capture
because<00:02:22.189> this<00:02:22.400> is<00:02:22.489> the<00:02:22.579> light<00:02:22.730> that<00:02:22.879> reflects
because this is the light that reflects
because this is the light that reflects
off<00:02:23.510> of<00:02:23.569> the<00:02:23.870> first<00:02:24.109> sir
off of the first sir
off of the first sir
the<00:02:24.650> skin<00:02:24.890> doesn't<00:02:25.130> get<00:02:25.400> underneath<00:02:25.730> the
the skin doesn't get underneath the
the skin doesn't get underneath the
translucent<00:02:26.780> layers<00:02:26.960> of<00:02:27.230> the<00:02:27.320> skin<00:02:27.590> and<00:02:27.740> blur
translucent layers of the skin and blur
translucent layers of the skin and blur
out<00:02:28.130> and<00:02:28.730> as<00:02:28.790> a<00:02:28.880> result<00:02:29.180> it's<00:02:29.330> a<00:02:29.420> very<00:02:29.660> good<00:02:29.900> cue
out and as a result it's a very good cue
out and as a result it's a very good cue
to<00:02:30.770> the<00:02:30.860> detailed<00:02:31.280> shape<00:02:31.640> of<00:02:31.880> the<00:02:32.270> skin<00:02:32.570> pore
to the detailed shape of the skin pore
to the detailed shape of the skin pore
structure<00:02:33.290> and<00:02:33.470> all<00:02:33.560> the<00:02:33.740> fine<00:02:33.980> wrinkles<00:02:34.190> that
structure and all the fine wrinkles that
structure and all the fine wrinkles that
all<00:02:34.640> of<00:02:34.790> us<00:02:34.880> have<00:02:35.060> the<00:02:35.510> things<00:02:35.720> that<00:02:35.870> actually
all of us have the things that actually
all of us have the things that actually
make<00:02:36.350> us<00:02:36.470> look<00:02:36.650> like<00:02:36.830> real<00:02:37.040> humans<00:02:37.460> so<00:02:37.970> if<00:02:38.090> we
make us look like real humans so if we
make us look like real humans so if we
use<00:02:38.450> information<00:02:38.690> that<00:02:39.260> comes<00:02:39.470> off<00:02:39.620> of<00:02:39.650> the
use information that comes off of the
use information that comes off of the
specular<00:02:40.280> reflection<00:02:40.820> we<00:02:41.690> can<00:02:41.870> go<00:02:41.990> from<00:02:42.170> a
specular reflection we can go from a
specular reflection we can go from a
traditional<00:02:42.800> face<00:02:43.040> scan<00:02:43.370> that<00:02:43.910> might<00:02:44.120> have
traditional face scan that might have
traditional face scan that might have
the<00:02:44.840> gross<00:02:45.110> contours<00:02:45.800> of<00:02:45.860> the<00:02:46.040> face<00:02:46.220> and<00:02:46.460> the
the gross contours of the face and the
the gross contours of the face and the
basic<00:02:46.910> shape<00:02:47.120> and<00:02:47.270> augment<00:02:48.170> it<00:02:48.320> with
basic shape and augment it with
basic shape and augment it with
information<00:02:49.160> that<00:02:49.670> puts<00:02:49.910> in<00:02:50.150> all<00:02:50.420> of<00:02:50.600> that
information that puts in all of that
information that puts in all of that
skin<00:02:51.050> pore<00:02:51.290> structure<00:02:51.710> and<00:02:51.920> fine<00:02:52.190> wrinkles
skin pore structure and fine wrinkles
skin pore structure and fine wrinkles
and<00:02:52.850> even<00:02:53.270> more<00:02:53.480> importantly<00:02:54.020> since<00:02:54.290> this<00:02:54.440> is
and even more importantly since this is
and even more importantly since this is
a<00:02:54.620> photometric<00:02:55.130> process<00:02:55.700> that<00:02:55.880> only<00:02:56.060> takes
a photometric process that only takes
a photometric process that only takes
three<00:02:56.540> seconds<00:02:56.960> to<00:02:57.110> capture<00:02:57.290> we<00:02:58.070> can<00:02:58.220> shoot
three seconds to capture we can shoot
three seconds to capture we can shoot
Emily<00:02:58.940> in<00:02:59.170> just<00:03:00.170> part<00:03:00.380> of<00:03:00.440> an<00:03:00.560> afternoon<00:03:00.860> in
Emily in just part of an afternoon in
Emily in just part of an afternoon in
many<00:03:01.730> different<00:03:01.970> facial<00:03:02.300> poses<00:03:02.780> and
many different facial poses and
many different facial poses and
different<00:03:03.140> facial<00:03:03.260> expressions<00:03:03.920> so<00:03:04.160> here<00:03:04.700> you
different facial expressions so here you
different facial expressions so here you
can<00:03:04.910> see<00:03:05.060> her<00:03:05.180> moving<00:03:05.360> your<00:03:05.510> eyes<00:03:05.630> around
can see her moving your eyes around
can see her moving your eyes around
moving<00:03:06.230> her<00:03:06.320> mouth<00:03:06.350> around<00:03:06.710> and<00:03:07.100> these<00:03:07.520> were
moving her mouth around and these were
moving her mouth around and these were
actually<00:03:08.090> going<00:03:08.240> to<00:03:08.330> use<00:03:08.450> to<00:03:08.660> create<00:03:08.780> a<00:03:08.930> photo
actually going to use to create a photo
actually going to use to create a photo
real<00:03:09.440> digital<00:03:09.650> character<00:03:10.250> so<00:03:10.910> if<00:03:11.000> you<00:03:11.060> take<00:03:11.210> a
real digital character so if you take a
real digital character so if you take a
look<00:03:11.510> at<00:03:11.630> these<00:03:11.750> scans<00:03:12.410> that<00:03:12.680> we<00:03:12.800> have<00:03:12.980> of
look at these scans that we have of
look at these scans that we have of
Emily<00:03:13.640> you<00:03:14.150> can<00:03:14.300> see<00:03:14.450> that<00:03:14.630> the<00:03:14.720> human<00:03:15.020> face
Emily you can see that the human face
Emily you can see that the human face
does<00:03:15.590> an<00:03:15.800> enormous<00:03:16.280> amount<00:03:16.460> of<00:03:16.610> amazing
does an enormous amount of amazing
does an enormous amount of amazing
things<00:03:17.630> as<00:03:18.020> it<00:03:18.560> goes<00:03:18.710> into<00:03:19.010> different<00:03:19.100> facial
things as it goes into different facial
things as it goes into different facial
expressions<00:03:20.240> you<00:03:20.810> can<00:03:20.840> see<00:03:21.200> things<00:03:21.500> not<00:03:22.460> only
expressions you can see things not only
expressions you can see things not only
the<00:03:22.790> base<00:03:22.970> shape<00:03:23.180> changes<00:03:23.690> but<00:03:23.900> all<00:03:24.170> sorts<00:03:24.410> of
the base shape changes but all sorts of
the base shape changes but all sorts of
different<00:03:24.710> skin<00:03:25.130> buckling<00:03:25.550> and<00:03:25.700> skin
different skin buckling and skin
different skin buckling and skin
wrinkling<00:03:26.240> occurs<00:03:26.660> you<00:03:27.410> can<00:03:27.590> see<00:03:27.830> that<00:03:28.070> the
wrinkling occurs you can see that the
wrinkling occurs you can see that the
skin<00:03:29.060> pore<00:03:29.300> structure<00:03:29.750> changes<00:03:30.200> enormous<00:03:30.740> Lee
skin pore structure changes enormous Lee
skin pore structure changes enormous Lee
from<00:03:31.610> stretched<00:03:32.209> skin<00:03:32.450> pores<00:03:32.920> to<00:03:33.920> the<00:03:34.040> regular
from stretched skin pores to the regular
from stretched skin pores to the regular
skin<00:03:34.670> texture<00:03:35.209> you<00:03:35.720> can<00:03:35.750> see<00:03:36.110> the<00:03:36.260> furrows<00:03:36.590> in
skin texture you can see the furrows in
skin texture you can see the furrows in
the<00:03:36.890> brow<00:03:37.070> and<00:03:37.250> how<00:03:37.370> the<00:03:37.430> microstructure
the brow and how the microstructure
the brow and how the microstructure
changes<00:03:38.630> there<00:03:38.780> you<00:03:38.900> can<00:03:39.020> see<00:03:39.110> muscles
changes there you can see muscles
changes there you can see muscles
pulling<00:03:39.890> down<00:03:40.220> it<00:03:40.430> flesh<00:03:40.730> to<00:03:40.970> bring<00:03:41.120> our
pulling down it flesh to bring our
pulling down it flesh to bring our
eyebrows<00:03:41.630> down<00:03:41.930> our<00:03:42.350> muscles<00:03:42.740> bulging<00:03:43.010> in<00:03:43.340> her
eyebrows down our muscles bulging in her
eyebrows down our muscles bulging in her
forehead<00:03:43.670> when<00:03:44.030> she<00:03:44.209> winces<00:03:44.750> like<00:03:44.930> that<00:03:45.140> in
forehead when she winces like that in
forehead when she winces like that in
addition<00:03:46.280> to<00:03:46.370> this<00:03:46.459> kind<00:03:46.700> of<00:03:46.730> high-resolution
addition to this kind of high-resolution
addition to this kind of high-resolution
geometry<00:03:48.110> since<00:03:48.500> it's<00:03:48.680> all<00:03:48.800> captured<00:03:49.040> with
geometry since it's all captured with
geometry since it's all captured with
cameras<00:03:49.730> we<00:03:49.910> get<00:03:50.030> a<00:03:50.060> great<00:03:50.330> texture<00:03:50.840> map<00:03:51.110> to
cameras we get a great texture map to
cameras we get a great texture map to
use<00:03:51.410> for<00:03:51.620> the<00:03:51.680> face<00:03:51.860> and<00:03:52.370> by<00:03:52.490> looking<00:03:52.730> at<00:03:52.820> how
use for the face and by looking at how
use for the face and by looking at how
the<00:03:53.030> different<00:03:53.120> color<00:03:53.690> channels<00:03:53.989> of<00:03:54.320> the
the different color channels of the
the different color channels of the
illumination<00:03:55.010> the<00:03:55.160> red<00:03:55.370> and<00:03:55.580> the<00:03:55.640> green<00:03:55.820> and
illumination the red and the green and
illumination the red and the green and
the<00:03:56.030> blue<00:03:56.270> diffused<00:03:57.200> the<00:03:57.350> light<00:03:57.500> differently
the blue diffused the light differently
the blue diffused the light differently
we<00:03:58.400> can<00:03:58.520> come<00:03:58.670> up<00:03:58.820> with<00:03:58.940> a<00:03:58.970> way<00:03:59.180> of<00:03:59.360> conveying
we can come up with a way of conveying
we can come up with a way of conveying
the<00:04:00.110> skin<00:04:00.590> and<00:04:00.739> the<00:04:00.800> computer<00:04:01.190> that<00:04:01.550> instead
the skin and the computer that instead
the skin and the computer that instead
of<00:04:01.970> looking<00:04:02.239> like<00:04:02.560> plaster<00:04:03.560> mannequin
of looking like plaster mannequin
of looking like plaster mannequin
actually<00:04:04.670> looks<00:04:04.910> like<00:04:05.030> it's<00:04:05.239> made<00:04:05.450> out<00:04:05.480> of
actually looks like it's made out of
actually looks like it's made out of
living<00:04:06.020> human<00:04:06.800> flesh
living human flesh
living human flesh
and<00:04:07.670> this<00:04:08.150> is<00:04:08.270> what<00:04:08.420> we<00:04:08.540> used<00:04:08.900> to<00:04:09.410> give<00:04:09.560> to<00:04:09.830> the
and this is what we used to give to the
and this is what we used to give to the
company<00:04:10.310> image<00:04:10.520> metrics<00:04:11.090> to<00:04:11.540> create<00:04:11.870> a<00:04:12.080> rigged
company image metrics to create a rigged
company image metrics to create a rigged
digital<00:04:13.489> version<00:04:13.880> of<00:04:14.000> Emily<00:04:14.360> we're<00:04:14.630> just
digital version of Emily we're just
digital version of Emily we're just
seeing<00:04:14.959> the<00:04:15.050> coarse<00:04:15.350> scale<00:04:15.590> geometry<00:04:16.160> here
seeing the coarse scale geometry here
seeing the coarse scale geometry here
but<00:04:16.520> they<00:04:16.609> basically<00:04:16.850> created<00:04:17.540> a<00:04:17.630> digital
but they basically created a digital
but they basically created a digital
puppet<00:04:18.380> of<00:04:18.500> her<00:04:18.739> where<00:04:19.340> you<00:04:19.549> can<00:04:19.669> pull<00:04:19.880> on
puppet of her where you can pull on
puppet of her where you can pull on
these<00:04:20.150> various<00:04:20.540> strings<00:04:20.900> and<00:04:21.290> it<00:04:21.650> actually
these various strings and it actually
these various strings and it actually
moves<00:04:22.190> her<00:04:22.430> face<00:04:22.640> in<00:04:22.880> ways<00:04:23.030> that<00:04:23.210> are
moves her face in ways that are
moves her face in ways that are
completely<00:04:23.419> consistent<00:04:23.780> with<00:04:24.440> the<00:04:24.620> scans
completely consistent with the scans
completely consistent with the scans
that<00:04:25.040> we<00:04:25.250> took<00:04:25.400> in<00:04:25.940> addition<00:04:26.300> to<00:04:26.360> the<00:04:26.450> coarse
that we took in addition to the coarse
that we took in addition to the coarse
scale<00:04:26.930> geometry<00:04:27.620> they<00:04:28.100> also<00:04:28.280> use<00:04:28.580> all<00:04:28.790> of<00:04:28.940> that
scale geometry they also use all of that
scale geometry they also use all of that
detail<00:04:29.360> to<00:04:29.870> create<00:04:30.230> a<00:04:30.260> set<00:04:30.470> of<00:04:30.500> what<00:04:30.710> are
detail to create a set of what are
detail to create a set of what are
called<00:04:30.830> displacement<00:04:31.730> maps<00:04:31.910> that<00:04:32.479> animate<00:04:32.930> as
called displacement maps that animate as
called displacement maps that animate as
well<00:04:35.200> these<00:04:36.200> are<00:04:36.380> the<00:04:36.410> displacement<00:04:37.070> maps
well these are the displacement maps
well these are the displacement maps
here<00:04:37.400> and<00:04:37.490> you<00:04:37.550> can
here and you can
here and you can
see<00:04:37.879> those<00:04:38.029> different<00:04:38.330> wrinkles<00:04:38.659> actually
see those different wrinkles actually
see those different wrinkles actually
show<00:04:39.199> up<00:04:39.349> as<00:04:39.529> she<00:04:39.889> animates<00:04:40.370> so<00:04:41.180> the<00:04:41.270> next
show up as she animates so the next
show up as she animates so the next
process<00:04:41.659> was<00:04:42.020> then<00:04:42.199> to<00:04:42.319> animate<00:04:42.800> her<00:04:42.919> we
process was then to animate her we
process was then to animate her we
actually<00:04:43.370> used<00:04:43.580> one<00:04:43.849> of<00:04:44.059> her<00:04:44.210> own
actually used one of her own
actually used one of her own
performances<00:04:45.110> to<00:04:45.319> provide<00:04:45.589> the<00:04:45.770> source<00:04:46.009> data
performances to provide the source data
performances to provide the source data
so<00:04:46.819> by<00:04:46.969> analyzing<00:04:47.180> this<00:04:47.809> video<00:04:48.139> with<00:04:48.349> computer
so by analyzing this video with computer
so by analyzing this video with computer
vision<00:04:49.009> techniques<00:04:49.159> they<00:04:49.550> were<00:04:49.639> able<00:04:49.729> to
vision techniques they were able to
vision techniques they were able to
drive<00:04:50.180> the<00:04:50.479> facial<00:04:50.900> rig<00:04:51.110> with<00:04:51.800> the
drive the facial rig with the
drive the facial rig with the
computer-generated<00:04:53.059> performance<00:04:53.779> so<00:04:53.900> what
computer-generated performance so what
computer-generated performance so what
you're<00:04:54.110> gonna<00:04:54.199> see<00:04:54.469> now<00:04:54.499> after<00:04:55.129> this<00:04:55.610> is<00:04:55.789> a
you're gonna see now after this is a
you're gonna see now after this is a
completely<00:04:56.240> photo<00:04:56.629> real<00:04:56.839> digital<00:04:57.050> face<00:04:57.409> we
completely photo real digital face we
completely photo real digital face we
can<00:04:57.710> turn<00:04:57.830> the<00:04:57.949> volume<00:04:57.979> up<00:04:58.339> a<00:04:58.460> little<00:04:58.610> bit<00:04:58.789> if
can turn the volume up a little bit if
can turn the volume up a little bit if
that's<00:04:58.999> available<00:04:59.479> which<00:04:59.569> metrics<00:05:00.110> is<00:05:00.379> a
that's available which metrics is a
that's available which metrics is a
marvelous<00:05:01.339> performance<00:05:01.580> driven<00:05:02.120> animation
marvelous performance driven animation
marvelous performance driven animation
company<00:05:03.229> we<00:05:03.979> specialize<00:05:04.669> in<00:05:04.699> high<00:05:05.210> quality
company we specialize in high quality
company we specialize in high quality
facial<00:05:06.319> animation<00:05:06.830> for<00:05:07.069> video<00:05:07.309> games<00:05:07.639> and
facial animation for video games and
facial animation for video games and
films<00:05:08.240> image<00:05:08.809> metrics<00:05:09.319> is<00:05:09.589> a<00:05:09.830> markerless
films image metrics is a markerless
films image metrics is a markerless
performance<00:05:11.180> driven<00:05:11.330> animation<00:05:11.990> company<00:05:12.439> we
performance driven animation company we
performance driven animation company we
specialize<00:05:13.490> in<00:05:14.149> high<00:05:14.449> quality<00:05:14.779> facial
specialize in high quality facial
specialize in high quality facial
animation<00:05:16.039> for<00:05:16.309> video<00:05:16.520> games<00:05:16.849> and<00:05:17.120> films<00:05:17.499> so
animation for video games and films so
animation for video games and films so
if<00:05:18.559> we<00:05:18.620> break<00:05:18.770> that<00:05:18.919> down<00:05:18.979> into<00:05:19.279> layers<00:05:19.520> here's
if we break that down into layers here's
if we break that down into layers here's
that<00:05:20.089> diffuse<00:05:20.449> component<00:05:20.599> we<00:05:20.990> saw<00:05:21.169> in<00:05:21.289> the
that diffuse component we saw in the
that diffuse component we saw in the
first<00:05:21.529> slide<00:05:21.729> here's<00:05:22.729> the<00:05:22.909> specular
first slide here's the specular
first slide here's the specular
component<00:05:23.839> animating<00:05:24.319> you<00:05:24.379> can<00:05:24.469> see<00:05:24.559> all<00:05:24.649> the
component animating you can see all the
component animating you can see all the
wrinkles<00:05:25.129> happening<00:05:25.550> there<00:05:25.729> and<00:05:26.059> there's<00:05:27.050> the
wrinkles happening there and there's the
wrinkles happening there and there's the
underlying<00:05:27.559> wireframe<00:05:28.159> mesh<00:05:30.669> and<00:05:31.669> that's
underlying wireframe mesh and that's
underlying wireframe mesh and that's
Emily<00:05:32.539> herself<00:05:32.830> now<00:05:33.830> where<00:05:34.159> are<00:05:34.219> we<00:05:34.279> going
Emily herself now where are we going
Emily herself now where are we going
with<00:05:34.669> this<00:05:34.819> here<00:05:35.029> we've<00:05:35.749> actually<00:05:35.960> gone<00:05:36.319> and
with this here we've actually gone and
with this here we've actually gone and
gone<00:05:37.039> a<00:05:37.069> little<00:05:37.219> bit<00:05:37.370> before<00:05:37.520> beyond<00:05:38.149> light
gone a little bit before beyond light
gone a little bit before beyond light
Stage<00:05:38.629> five<00:05:38.839> this<00:05:39.050> is<00:05:39.229> light<00:05:39.409> stage<00:05:39.680> six<00:05:39.979> and
Stage five this is light stage six and
Stage five this is light stage six and
we're<00:05:40.249> looking<00:05:40.490> at<00:05:40.580> taking<00:05:40.879> this<00:05:40.939> technology
we're looking at taking this technology
we're looking at taking this technology
and<00:05:41.509> applying<00:05:41.599> it<00:05:41.959> to<00:05:42.020> whole<00:05:42.259> human<00:05:42.680> bodies
and applying it to whole human bodies
and applying it to whole human bodies
this<00:05:43.249> is<00:05:43.399> Bruce<00:05:43.669> LOM<00:05:43.909> and<00:05:43.939> one<00:05:44.180> of<00:05:44.300> our
this is Bruce LOM and one of our
this is Bruce LOM and one of our
researchers<00:05:44.870> in<00:05:45.020> the<00:05:45.080> group<00:05:45.259> who<00:05:45.740> graciously
researchers in the group who graciously
researchers in the group who graciously
agreed<00:05:46.370> to<00:05:46.550> get<00:05:46.729> captured<00:05:47.209> running<00:05:47.569> in<00:05:47.689> the
agreed to get captured running in the
agreed to get captured running in the
light<00:05:47.930> stage<00:05:48.199> and<00:05:48.649> let's<00:05:48.830> take<00:05:48.919> a<00:05:48.979> look<00:05:49.159> at<00:05:49.249> a
light stage and let's take a look at a
light stage and let's take a look at a
computer-generated<00:05:49.819> version<00:05:50.509> of<00:05:50.659> Bruce
computer-generated version of Bruce
computer-generated version of Bruce
running<00:05:51.830> in<00:05:51.919> a<00:05:51.979> new<00:05:52.099> environment
and<00:05:59.060> thank<00:05:59.240> you<00:05:59.270> very<00:05:59.360> much
Paul Debevec, TED, TEDx, TEDxUSC, USC, computer, graphics, photo, realism, TED-Ed, TED, Ed, TEDEducation
