Definition: What are the artistic characteristics of fire? How can we segment it?

Visual Characteristics of Fire

This post is a first pass on my attempt to define what precisely defines fire from an artistic perspective, which will be essential for our group's Advanced Computer Vision project. This is simply to get me thinking about fire. There are many articles that will tell you what fire is scientifically, and our code (which will probably be interfacing with Houdini) will certainly not be ignoring that -- But there are no easy answers in the rather inexact science of what precisely a fire is, visually speaking. 

Here are a list of attributes I have come up with. I may modify this later based on what we come up with as a group: 

• Fire is usually comparatively bright with respect to its background. This is not always the case, especially with fires pictured in the daytime. Furthermore, consuming wildfires that comprise the majority of the space in a shot may be somewhat difficult to segment as they can be so large that there isn't much of a visible background at all. My segmentation lab in Computer Vision was rather finicky when coming across images with narrow foreground aspects -- such as the thin stem of a rose. Or the thin, licking flames of a fire. Though admittedly that algorithm needs some work, I do anticipate the fire to be somewhat difficult to separate from its background. 
• The center of a fire is usually significantly bright, with a small amount of light attenuation that decreases until the background is reached. In contrast, a spotlight has a ton of light attenuation, which is quite noticeable. Whereas a fire is usually pretty harsh against its background -- but again, there is attenuation of its light. 
• A spiraling upwards shape. Not always, but a fire tends to narrow as it reaches skyward -- and it almost always reaches skyward. 
• Lots of thin and narrow tendrils, in addition to a potentially larger base 
• Fires are generally orange in their base color, with yellow, red, and sometimes electric blue present as well. Plus gray, for the smoke. 
• Color that varies based on temperature, and
• Smoke. But that's not always visible in images of fire. 

Here are some cool fire images I found online a long time ago, can't remember where they're from (not mine): 

Segmenting Fire

Choi et. al. discuss a method for semantic fire segmentation using a convolutional neural network. This paper has practical applications for rapid fire detection, in order to allow firefighters to respond to threats at a more immediate rate, and in locations where people aren't necessarily around. I haven't perused the algorithm in detail, but it appears to be adding two additional convolution layers to the architecture, which aims to enhance the "middle-skip" connection. 

The results are very good, especially when compared with other fire segmentation work done previously. However, it appears to experience some difficulty when the images of fire are rotated. The architecture is based on a similar architecture known as FusionNet, but has some underlying mathematical differences, which are addressed in the paper. More than anything, Choi et. al. teach us that effective fire segmentation, while not perfect, is entire possible to implement, and is certainly an enhancement over previous methods. 

I suspect this will be a good jumping-off point for our project.