documentation outlines coming together slowly
@@ -0,0 +1,63 @@
|
||||
# Colormap Examples
|
||||
Keeping everything the same, but changing the color map to demonstrate visual differences.
|
||||
|
||||
## understanding Colormaps
|
||||
These apply a color gradient to the greyscale saliency values. Low values (less important) get one end of the gradient, high values (more important) get the other.
|
||||
inferno — Black → purple → orange → yellow. The default. Dramatic, high contrast.
|
||||
plasma — Purple → pink → yellow. Vibrant and synthetic-looking. Feels digital, almost neon.
|
||||
magma — Black → deep purple → pink-white. Darker and contrast than inferno.
|
||||
viridis — Purple → teal → yellow-green. The standard scientific colormap, designed to be perceptually uniform.
|
||||
hot — Black → red → orange → white. Classic heat map / thermal camera.
|
||||
cool — Cyan → magenta. Cold, clinical.
|
||||
spring — Magenta → yellow. Neon, synthetic, light greens.
|
||||
copper — Black → dark brown → copper-gold. Archival, almost daguerreotype-y.
|
||||
twilight — purple → white → orange → purple. Atmospheric brown.
|
||||
ocean — Black-blue → cyan → white. Deep and watery blue
|
||||
|
||||
## What is it?
|
||||
|
||||
Model_name: vgg16
|
||||
method: guided IG
|
||||
composition: inverse
|
||||
colormap: THIS IS WHAT CHANGES
|
||||
intensity: 0.75
|
||||
contrast: 1.5
|
||||
threshold: 0.25
|
||||
class_index: -1
|
||||
input_size: 224
|
||||
uses_smooth_grad: true
|
||||
snoothgrad_samples: 25
|
||||
|
||||
|
||||
## listed
|
||||
|
||||
|
||||
cool
|
||||

|
||||
|
||||
hot
|
||||

|
||||
|
||||
copper
|
||||

|
||||
|
||||
inferno
|
||||

|
||||
|
||||
magma
|
||||

|
||||
|
||||
ocean
|
||||

|
||||
|
||||
plasma
|
||||

|
||||
|
||||
spring
|
||||

|
||||
|
||||
twilight
|
||||

|
||||
|
||||
viridis
|
||||

|
||||
|
After Width: | Height: | Size: 108 KiB |
|
After Width: | Height: | Size: 102 KiB |
|
After Width: | Height: | Size: 113 KiB |
|
After Width: | Height: | Size: 120 KiB |
|
After Width: | Height: | Size: 103 KiB |
|
After Width: | Height: | Size: 113 KiB |
|
After Width: | Height: | Size: 114 KiB |
|
After Width: | Height: | Size: 108 KiB |
|
After Width: | Height: | Size: 118 KiB |
|
After Width: | Height: | Size: 119 KiB |
@@ -1,7 +1,18 @@
|
||||
# Methods Examples
|
||||
Keeping everything the same, but changing the model to demonstrate visual differences.
|
||||
|
||||
# Composition Examples
|
||||
Keeping everything the same, but changing the composition to demonstrate visual differences.
|
||||
|
||||
## Understanding compostion modes
|
||||
These control how the saliency map is composited back onto the original image. This is where the artistic intent lives.
|
||||
isolation — The default and most conceptually pointed for your project. Salient regions show in full color using the chosen colormap; everything else is converted to greyscale. The effect: the machine's gaze burns in color while everything it ignores fades to grey. Very direct visual commentary on algorithmic attention.
|
||||
overlay — Classic heatmap blend. The colormap is alpha-composited over the original at the intensity level. Familiar, readable, less subversive.
|
||||
spotlight — The salient regions are bright, everything else darkens dramatically. Feels like a searchlight or surveillance camera — forensic, clinical.
|
||||
ghost — The original image fades to 25% opacity and the saliency map glows over it. The subject becomes spectral, haunted by its own classification. Good for an uncanny effect.
|
||||
invert — Shows what the algorithm ignores instead of what it sees. The saliency map is flipped before compositing. Conceptually this is the most subversive mode — it visualizes the blind spots, the parts of a face that are invisible to the machine.
|
||||
cutout — Hard binary mask. Pixels above the threshold value show the original image; everything below shows a dim version of the heatmap. Creates a stark, graphic effect — the subject literally cut out by the algorithm's attention boundary.
|
||||
multiply — Photoshop-style multiply blend. Attention dims non-salient areas. Produces a dark, moody result where ignored regions sink into shadow.
|
||||
screen — Photoshop-style screen blend. Attention brightens salient regions. Lighter and more luminous than multiply — glowing rather than shadowed.
|
||||
triptych — Outputs three panels side by side: original | saliency map alone | composite. Designed for gallery or exhibition display — shows the full chain of the visualization in one image.
|
||||
mask_only — Just the colored saliency map with no original image. Pure algorithmic output, no human context.
|
||||
|
||||
## What is it?
|
||||
|
||||
|
||||
@@ -0,0 +1,85 @@
|
||||
# Colormap Examples
|
||||
Keeping everything the same, but changing the color map to demonstrate visual differences.
|
||||
|
||||
|
||||
Other Parameters
|
||||
intensity — How strongly the saliency visualization is blended onto the original. 0 = invisible, 1 = full. Most compositions look best between 0.6–0.85.
|
||||
contrast (gamma) — Applies a power function to the saliency values before visualization. Values above 1.0 compress the low end and expand the high end — making the most salient regions pop dramatically while less important regions fade further. Cranking this to 2.5–3.0 produces extreme, high-drama results where only the very top attention regions survive.
|
||||
threshold — Used primarily by isolation and cutout modes. Sets the minimum saliency value to be considered "attended to." Higher values make the visualization sparser and more selective — only the regions the model is most confident about survive.
|
||||
class_index — ImageNet has 1000 classes. By default (-1) the node auto-detects what the model predicted and visualizes attention for that class. You can override this to force the visualization to show what the model would attend to if it were trying to classify the image as a specific thing — for example, forcing class 207 (golden retriever) on a human portrait produces attention patterns for a question the model was never meant to answer, which has interesting artistic implications.
|
||||
input_size — All these models were trained on fixed-size inputs. 224×224 is standard for most; InceptionV3 requires 299×299. The image is resized to this before processing, which is why the output appears at that resolution.
|
||||
use_smoothgrad — When enabled on top of any method (except SmoothGrad which already does this), applies the SmoothGrad averaging process to smooth the result. Generally produces more aesthetically pleasing outputs at the cost of being slower.
|
||||
smoothgrad_samples — How many noisy samples to average when SmoothGrad is active. 25 is a good balance; 50+ produces very smooth results but takes proportionally longer.
|
||||
|
||||
|
||||
## What is it?
|
||||
|
||||
Model_name: vgg16
|
||||
method: guided IG
|
||||
composition: inverse
|
||||
colormap: magma
|
||||
[the following IS WHAT CHANGES]
|
||||
default values:
|
||||
|
||||
intensity: 0.75
|
||||
contrast: 1.5
|
||||
threshold: 0.25
|
||||
class_index: -1
|
||||
input_size: 224
|
||||
uses_smooth_grad: true
|
||||
snoothgrad_samples: 25
|
||||
|
||||
|
||||
## listed out
|
||||
should explain why i picked the default values i did.
|
||||
|
||||
### Intensity
|
||||
|
||||
intensity = 0.5
|
||||
(lower)
|
||||

|
||||
|
||||
intensity = 1.0
|
||||
(higher)
|
||||

|
||||
|
||||
### Contrast
|
||||
|
||||
contrast = 1.0
|
||||
(lower)
|
||||

|
||||
|
||||
contrast = 2.0
|
||||
(higher)
|
||||

|
||||
|
||||
### threshold
|
||||
threshold = 0.0
|
||||
(lower)
|
||||

|
||||
|
||||
threshold = 0.5
|
||||
(higher)
|
||||

|
||||
|
||||
### class index
|
||||
class index = 1
|
||||
(a little change)
|
||||

|
||||
|
||||
class index = 10
|
||||
(a larger change)
|
||||

|
||||
|
||||
### input size
|
||||
Didn't adjust this because most models require this size (224) to run properly.
|
||||
|
||||
### uses smooth grad
|
||||
Without (uses smooth grad = false)
|
||||

|
||||
|
||||
|
||||
|
||||
### smooth grad samples
|
||||
samples = 50
|
||||

|
||||
|
After Width: | Height: | Size: 76 KiB |
|
After Width: | Height: | Size: 77 KiB |
|
After Width: | Height: | Size: 61 KiB |
|
After Width: | Height: | Size: 72 KiB |
|
After Width: | Height: | Size: 70 KiB |
|
After Width: | Height: | Size: 70 KiB |
|
After Width: | Height: | Size: 68 KiB |
|
After Width: | Height: | Size: 82 KiB |
|
After Width: | Height: | Size: 70 KiB |
|
After Width: | Height: | Size: 70 KiB |
@@ -1,6 +1,14 @@
|
||||
# Methods Examples
|
||||
Keeping everything the same, but changing the model to demonstrate visual differences.
|
||||
|
||||
## Understanding methods
|
||||
These are the mathematical techniques used to ask "what pixels mattered to this model's decision?" Each has a different visual character.
|
||||
Vanilla Gradients — The simplest approach: take the derivative of the output with respect to each input pixel. Fast and raw. The result is noisy and looks almost like static or grain — shows every pixel that had any influence at all, including spurious ones. Visually the most aggressive and chaotic.
|
||||
SmoothGrad — Takes Vanilla Gradients but runs it many times with small random noise added to the image each time, then averages the results. This cancels out the noise and reveals more stable patterns. Visually smoother and more "reliable" looking than vanilla, but still texture-rich. The smoothgrad_samples parameter controls how many noisy runs are averaged — more = smoother but slower.
|
||||
Integrated Gradients — Instead of just looking at gradients at the input image, this traces a path from a blank black image to your actual image and accumulates gradients along that path. This gives it a strong theoretical foundation (it satisfies mathematical axioms about attribution). Produces clean, smooth maps that highlight meaningful regions rather than just any active pixels. The ig_steps parameter controls how many steps along that path — more = more accurate but slower. This is usually the best starting method.
|
||||
Blur IG — A variant of Integrated Gradients where instead of interpolating from black to the image, it interpolates from a heavily blurred version to the sharp image. This tends to produce maps focused on edges and structural details rather than color/texture. Produces a different aesthetic — more architectural-looking.
|
||||
Guided IG — Another IG variant that uses gradient information to choose a smarter integration path, concentrating steps where the gradients are largest. Produces sharper, higher-contrast attribution maps with more defined edges. Often the most visually striking for portraits.
|
||||
XRAI — Completely different approach. Instead of pixel-level gradients, it segments the image into regions and attributes importance to each region as a whole. Produces a patchwork/segmented look — almost like stained glass or a heat map made of irregular shapes. Very different aesthetic from the gradient methods. Uses algorithm="fast" by default; changing to "full" in the code gives higher quality but is much slower.
|
||||
|
||||
|
||||
## What is it?
|
||||
|
||||
@@ -0,0 +1,36 @@
|
||||
# Model Examples
|
||||
Keeping everything the same, but changing the model to demonstrate visual differences.
|
||||
|
||||
## Understanding Models
|
||||
Models
|
||||
The model is the "brain" being interrogated — it's a pretrained ImageNet classifier whose internal decision-making we're visualizing. Different models have different "personalities" in how they look at images.
|
||||
|
||||
need to explain why these models, where they came from. explain reasoning behind choices
|
||||
|
||||
VGG16 / VGG19 — Old, deep, sequential architecture from 2014. For portraits, these are the most interesting because they're blunt and spatially crude. The attention patterns feel almost aggressive — big, blocky regions. Good for the "surveillance" feeling you're going for. VGG16 is the default for a reason.
|
||||
ResNet50 / ResNet101 — More modern, uses skip connections. Produces cleaner, more distributed attention. Less visually dramatic than VGG but more "honest" about what it's actually classifying.
|
||||
<!-- EfficientNet-B0 — Very modern, highly optimized. Produces fine-grained, sparse attention maps. Interesting for showing how efficiently a model can locate a subject with minimal "gaze." -->
|
||||
DenseNet121 — Dense connectivity means attention is highly distributed across the image. Produces diffuse, almost painterly saliency maps.
|
||||
MobileNetV3 — Designed for mobile devices, so it's aggressively compressed. Produces the sparsest, most stripped-down attention — shows only the absolute minimum the model needs to make a decision.
|
||||
InceptionV3 — Uses parallel convolutions at multiple scales. Produces multi-scale attention that can look almost fractal. Requires input_size 299 instead of 224.
|
||||
|
||||
## What is it?
|
||||
|
||||
Model_name: THIS IS WHAT CHANGES
|
||||
method: guided IG
|
||||
composition: pure_white
|
||||
colormap: magma
|
||||
intensity: 0.75
|
||||
contrast: 1.5
|
||||
threshold: 0.25
|
||||
class_index: -1
|
||||
input_size: 224
|
||||
uses_smooth_grad: true
|
||||
snoothgrad_samples: 25
|
||||
|
||||
|
||||
## listed out
|
||||
|
||||
|
||||
cool
|
||||

|
||||
@@ -36,7 +36,8 @@ IMAGENET_STD = np.array([0.229, 0.224, 0.225], dtype=np.float32)
|
||||
SUPPORTED_MODELS = [
|
||||
"vgg16", "vgg19", "resnet50", "resnet101",
|
||||
"inception_v3", "mobilenet_v3_large",
|
||||
"efficientnet_b0", "densenet121",
|
||||
# "efficientnet_b0",
|
||||
"densenet121",
|
||||
]
|
||||
|
||||
_model_cache: dict = {}
|
||||
@@ -72,7 +73,7 @@ def _load_model(model_name: str):
|
||||
"vgg16": tv.VGG16_Weights.DEFAULT,
|
||||
"vgg19": tv.VGG19_Weights.DEFAULT,
|
||||
"mobilenet_v3_large": tv.MobileNet_V3_Large_Weights.DEFAULT,
|
||||
"efficientnet_b0": tv.EfficientNet_B0_Weights.DEFAULT,
|
||||
# "efficientnet_b0": tv.EfficientNet_B0_Weights.DEFAULT,
|
||||
"densenet121": tv.DenseNet121_Weights.DEFAULT,
|
||||
}
|
||||
model = constructor(weights=weight_map.get(model_name))
|
||||
|
||||