They're here! More Muvipix.com Guides by Steve Grisetti!
The Muvipix.com Guides to Premiere & Photoshop Elements 2024
As well as The Muvipix.com Guide to CyberLink PowerDirector 21
Because there are stories to tell
muvipix.com

Face Centred Gradient Fade Using AI

Video / Image editing, advanced techniques, computer settings, third party software, shortcuts, workarounds ... share your tips and tricks here.

Face Centred Gradient Fade Using AI

Postby Chris B » Thu Dec 31, 2020 4:59 am

I'm always looking for something new to do for the yearly slide show. As I mentioned before I've got a script that can be used render the same file over and over again replacing the pictures in it to make a whole show (viewtopic.php?f=93&t=11347 - wow was that really 2012...). Recently I used this with some gradient fades based on noise textures.

These worked well but were just randomly applied at the transitions. However sometimes the fade would start with a point of interest in the picture - say a person or place - and that looked good. So it got me thinking and I've built a python script to automatically create a gradient based on the position of the face(s) in a photo. A video is worth a million words so here's an example:



You can see that the fade is centred on the face or faces in the photos. The gradient for the change from the first to second face is this:
g2small.png

The black part, where the fade starts, is offset in the image to the position of the second face.

Now for the science bit...
Technically what I've done is use a package called MTCNN (https://github.com/ipazc/mtcnn) which uses the tensorflow machine learning platform to do the face recognition. This returns a list of bounding boxes that are what it detects as faces within the photo. I've then written a routine to generate a circular gradient that centres around these boxes and adds a little bit of "noise" to it to make the fade more interesting. Whilst it sounds complicated all of the heavy lifting is done using a library so once it's set up (which is quite a fussy process) there is actually quite a small amount of code required. Rinse and repeat for the entire slide show :)
You do not have the required permissions to view the files attached to this post.
Intel Core i7 8700 - 32GB DDR4 - 500GB Evo 970 SSD - 3+2 TB HDD - GTX 1080- MSI Z370 Pro - Win10 64 bit - Cannon HV30 (PAL) - Sony A6000 - GoPro 3 Black
User avatar
Chris B
Moderator
Moderator
 
Posts: 819
Joined: Tue Apr 24, 2007 6:04 pm
Location: UK

Re: Face Centred Gradient Fade Using AI

Postby Steve Grisetti » Thu Dec 31, 2020 7:58 am

This is an interesting effect, Chris. What program did you edit this slideshow on?
HP Envy with 2.9/4.4 ghz i7-10700 and 16 gig of RAM running Windows 11 Pro
User avatar
Steve Grisetti
Super Moderator
Super Moderator
 
Posts: 14439
Joined: Sat Feb 17, 2007 5:11 pm
Location: Milwaukee, Wisconsin

Re: Face Centred Gradient Fade Using AI

Postby Chris B » Thu Dec 31, 2020 8:23 am

The slide show was rendered with Vegas. I actually cheated slightly for the demo and just put together a very small project. When I do it for real there will be a template file in Vegas and the script will swap them out and start a render. You then end up with a lot of small video files that just need dropping into another project for the final render/add music/titles and the like. For efficiency I'll probably have about 5 photos in the mini project otherwise it spends more time starting Vegas than rendering the files.
Intel Core i7 8700 - 32GB DDR4 - 500GB Evo 970 SSD - 3+2 TB HDD - GTX 1080- MSI Z370 Pro - Win10 64 bit - Cannon HV30 (PAL) - Sony A6000 - GoPro 3 Black
User avatar
Chris B
Moderator
Moderator
 
Posts: 819
Joined: Tue Apr 24, 2007 6:04 pm
Location: UK

Re: Face Centred Gradient Fade Using AI

Postby Bob » Thu Dec 31, 2020 3:46 pm

Very nice, Chris!
User avatar
Bob
Moderator
Moderator
 
Posts: 5925
Joined: Wed Feb 21, 2007 4:49 am
Location: Southern California, USA

Re: Face Centred Gradient Fade Using AI

Postby Peru » Fri Jan 01, 2021 1:18 pm

I like it. :tup:
I'm going to try to duplicate that with Premiere Pro and/or After Effects the next time I do a slide show.
User avatar
Peru
Moderator
Moderator
 
Posts: 3687
Joined: Wed Oct 17, 2007 12:34 pm
Location: Peru, NY, USA

Re: Face Centred Gradient Fade Using AI

Postby ed » Sat Jan 02, 2021 9:10 am

That’s really cool. Have you worked with the Pillow library? I used it to create 2000 thumbnails with a logo overlay from a directory of photos for work. It was pretty fast.

[ Post made via Mobile Device ] Image
User avatar
ed
Premiere Member
Premiere Member
 
Posts: 1586
Joined: Thu Feb 22, 2007 9:17 am

Re: Face Centred Gradient Fade Using AI

Postby Chris B » Sat Jan 02, 2021 9:18 am

Yes - I used pillow to generate the gradient - although I'm sure my Python is very sub optimal. I ought to go back and do it "properly" but it seems to work as is and It's only going to be a one off. The recognition takes the majority of the time.
Intel Core i7 8700 - 32GB DDR4 - 500GB Evo 970 SSD - 3+2 TB HDD - GTX 1080- MSI Z370 Pro - Win10 64 bit - Cannon HV30 (PAL) - Sony A6000 - GoPro 3 Black
User avatar
Chris B
Moderator
Moderator
 
Posts: 819
Joined: Tue Apr 24, 2007 6:04 pm
Location: UK


Return to Tips and Tricks 


Similar topics


Who is online

Users browsing this forum: No registered users and 8 guests

cron