Introduction to Neural Networks - code examples¶

BWXT Data Science Workforce Training Pilot
Companion to Chapter_Introduction_to_Neural_Networks.md

Run these examples after reading the introduction chapter. They cover perceptrons, activations, losses, and a single gradient-descent update using small numbers you can inspect by hand.

Setup¶

These examples use only the Python standard library plus pprint for readable dictionaries.

In [ ]:
import math
from pprint import pprint

print('Setup complete.')

Perceptrons and Neurons¶

A perceptron calculates a weighted sum plus a bias. A neuron usually applies an activation function to that score.

In [ ]:
x1 = 0.8
x2 = 0.4
x3 = 0.2

w1 = 0.5
w2 = -0.25
w3 = 0.75
bias = 0.1

score = w1 * x1 + w2 * x2 + w3 * x3 + bias
print('Weighted score:', score)
In [ ]:
def relu(x):
    return max(0, x)


neuron_output = relu(score)
print('Neuron output after ReLU:', neuron_output)

Activation Functions¶

These examples run the ReLU, sigmoid, and softmax ideas from the chapter.

In [ ]:
def relu(x):
    return max(0, x)


print(relu(-3))
print(relu(5))
In [ ]:
def sigmoid(x):
    return 1 / (1 + math.exp(-x))


print(sigmoid(0))
print(sigmoid(2))
In [ ]:
def softmax(scores):
    exponentials = [math.exp(score) for score in scores]
    total = sum(exponentials)
    return [value / total for value in exponentials]


class_scores = {
    'no_defect': 1.2,
    'porosity': 2.5,
    'crack': 0.4,
}

probabilities = softmax(list(class_scores.values()))
class_probabilities = dict(zip(class_scores.keys(), probabilities))

pprint(class_probabilities)
print('Predicted class:', max(class_probabilities, key=class_probabilities.get))

Loss Functions¶

These examples show simple binary cross entropy, mean squared error, and a use-case-driven class weighting choice.

In [ ]:
def binary_cross_entropy(y_true, y_pred):
    epsilon = 1e-12 # elinminate log(0)
    y_pred = min(max(y_pred, epsilon), 1 - epsilon)
    return -(y_true * math.log(y_pred) + (1 - y_true) * math.log(1 - y_pred))


print('Loss for true defective, predicted 0.95:', binary_cross_entropy(1, 0.95))
print('Loss for true defective, predicted 0.05:', binary_cross_entropy(1, 0.05))
In [ ]:
true_length = 5.0
predicted_length = 7.0

error = predicted_length - true_length
squared_error = error ** 2

print('Error:', error)
print('Squared error:', squared_error)
In [ ]:
class_weights = {
    'no_defect': 1.0,
    'porosity': 2.0,
    'undercut': 2.0,
    'crack': 5.0,
}

print('Class weights for rare critical defects:')
pprint(class_weights)

Gradient Descent¶

Backpropagation calculates gradients. Gradient descent uses those gradients to update weights.

In [ ]:
old_weight = 0.80
learning_rate = 0.10
gradient = 0.30

new_weight = old_weight - learning_rate * gradient

print('Old weight:', old_weight)
print('Gradient:', gradient)
print('Learning rate:', learning_rate)
print('New weight:', new_weight)