All Articles

Mask R-CNN with the TensorFlow C++/CUDA API: Automatic Rotoscoping

I made a Plug-in for Adobe Premiere/After Effects that uses the TensorFlow object detection framework to automatically detect objects and generate segmentation masks for them. Compiling the TensorFlow as a C++ library for Windows with CUDA functionality was quite a tedious task, but in the end I was able to build a plug-in using Mask R-CNN to generate quite acceptable masks for objects contained in the 90 COCO classes. On my notebook (with 1070 GTX) the performance was quite good, resulting in several processed frames per second. As the project was more a proof of concept, there are currently no plans of a commercial release (probably open source release soon).