Project
Automatic construction by robots
For my bachelor thesis I've created a procedural animation system that can create automatically animated scenes of two robots collaboratively constructing an object from fractured pieces (Given a any polygonal object as input). When I applied for an internship at robotics company Kuka in april 2019, I thought a nice animation of their logo (or its first letter) might make me stand out a bit more. This is the result.
ReadKinectFusion in the browser
in summer semester 2019 I worked on a state of the art report about geometry reconstruction fraom cameras for a university seminar. When I encountered quite fascinating techniques while getting familiar with the topic, I decided to implement one of them in WebGL / three.js to get some hands on experience. I implemented the truncated signed distance field based fusion method from the “KinectFusion” paper by Newcombe et al. I was able to implement the algorithm purely in vertex and fragment shaders, using an efficient 2d to 3d offscreen texture mapping for the parallelized computation.
ReadIce skating collision prediction
In this work, which started as a client project, I had the opportunity to deal with various interesting challenges: Finding a tracking architecture capable of tracking dozens of persons in real time, generating training data for a scenario where real footage would be unavailable during development and finally finding a way to rapidly find intersections between a large amount of curved trajectories for collision prediction.
ReadGPU Particles
This project is the result of a real time graphics course I took in winter semester 2018. We were tasked to create an automatically running demo scene and I implemented a massive particle system (>4 mio.) with inter particle-forces in OpenGL without the use of compute shaders. For the beginning I simulated a computation-heavy explosion vector field in an offline fluid simulation, which I then was able to use in a 3d texture for the initial particle advection. The scene is complemented by a lightmap based illumination system that reacts to the music, and omnidirectional shadow mapping.
ReadMask R-CNN with the TensorFlow C++/CUDA API: Automatic Rotoscoping
I made a Plug-in for Adobe Premiere/After Effects that uses the TensorFlow object detection framework to automatically detect objects and generate segmentation masks for them. Compiling the TensorFlow as a C++ library for Windows with CUDA functionality was quite a tedious task, but in the end I was able to build a plug-in using Mask R-CNN to generate quite acceptable masks for objects contained in the 90 COCO classes.
ReadReframe 360 Plugin
The „reframe360“ plugin is something I started a few weeks ago out of the wish to be able to edit footage recorded with my at the time freshly bought Gopro Fusion 360 degree camera in DaVinci Resolve. Also I found Gopros own reframing (reframing in this context means choosing a normal perspective camera angle within the 360 footage in editing) plugin, that is provided for free for the Adobe Premiere and After Effects, to lack a lot of functionality (even a simple rectilinear perspective projection is not implemented, only fisheye variants). Another motivation to start this project was to get some practice in GPGPU programming (CUDA and OpenCL).
ReadConvolutional neural network visualizer
In summer semester 2018 I attended the combined lecture/excercice „Visualization 2“ at my university, where we were assigned to implement a visualization based on one of various suggested papers, but we also had the possibility to implement an own idea if we preferred that. I wanted to create a 3d visualization of a Convolutional Neural Network showing all of the connections between layers of these models, to give a person new to this field a good visual overview of the interdependency. The implementation has been done in the game engine Unity.
ReadSynthetic pose estimation training
I created a setup to synthetically generate automatically annotated training images for pose estimation in the context of plate tracking.
ReadSiamese multi object tracking
During an internship at MoonVision GmbH I was tasked with researching the potential of siamese tracking for an internal project. I adapted an existing TensorFlow implementation to be able to handle multiple objects instead of just one. I also created the visualization of a heatmap displaying the location probability of the object to track. This required a deep adaption of the architecture, introducing a fourth dimension into the three dimensional computation graph, but provided valuable insight into the details of this powerful architecture.
ReadSynthetic snickers counting
I generated a synthetic dataset of photorealistic renderings containing a varying amount of Snickers chocolate bars, arranged in a realistic way by rigid body simulation. This dataset was then used to train a classifier counting the amount of Snickers in each image. I used this project when following introductory tutorials to the PyTorch library.
ReadBib recognition
Bib recognition is the task of identifying and reading the identification numbers that athletes wear during race events. Automating this task is valuable for sports photographers who often have to tag ten thousands of athletes for larger events. Initially the problem seems quite simple, but in many cases the numbers are partially occluded or heavily distorted, making a reliable solution quite difficult. This project served me as a practical example while learning to use tensorflow and over the time I developed quite a sophisticated multi stage pipeline to tackle the problem.
ReadTraining an inverted-pendulum stabilizing RNN by genetic algorithm
Inspired by a lecture about heuristic optimization I implemented various GA experiments in my go-to 3d package Houdini, where robust rigid-body simulations are possible. In this example I trained a recurrent neural network (RNN) to stabilize an inverted double pendulum. The loss function was integrating the distance to the target position over the course of a single episode. The complete RNN including activations (on shperes) and weights (on edges) is shown on the left, while on the right the current population can be seen, with the best performers closest to the camera.
Read