Types of Neural Networks Explained (Your First TensorFlow Model in Python)

Updated October 2026 | 17 min read

If you followed our neural network from scratch in Python notes, you built one network by hand — forward pass, backpropagation, the bias fix — and watched it learn XOR. Now it’s time to see the whole family. There isn’t one “neural network”; there are several types of neural networks, each shaped for a different kind of data, and knowing which one to reach for is half the skill.

This page does two things. First, it explains the main types — ANN, CNN, RNN, LSTM, Transformer and a couple of others — with what each is for and a real example. Second, it gets you building: the same XOR network from page 1, rewritten in 15 lines of TensorFlow/Keras, and then a real convolutional network that reads handwritten digits at ~99% accuracy. Tested code, real output, and the mistakes you’ll actually hit.

Quick Recap: What Every Neural Network Has in Common

No matter the type, every neural network is built from the same four parts you already know: layers of neurons, weights on the connections between them, a bias per neuron, and an activation function that adds non-linearity. Data goes in, flows forward through the layers, a loss is measured, and backpropagation nudges the weights. The different types differ in how the layers are wired — and that wiring is what makes one type good at images and another at text. If any of those four words is fuzzy, the from-scratch page explains each with real numbers.

The Main Types of Neural Networks

1. Feedforward Network (ANN / MLP)

What it is: The basic one. Data flows strictly forward — input → hidden layers → output — with every neuron connected to every neuron in the next layer. The network you built from scratch is exactly this.
Best for: Tabular data — rows and columns, like a spreadsheet.
Real example: Predicting house prices from size, location and age; deciding whether a loan application is risky.
Shape: input → [dense] → [dense] → output

2. Convolutional Neural Network (CNN)

What it is: Instead of connecting every pixel to every neuron, a CNN slides small filters across the image to detect edges, then textures, then shapes, then objects — building up from simple to complex. This makes it dramatically more efficient on images than a plain ANN.
Best for: Images and anything grid-shaped — photos, medical scans, video frames.
Real example: Face unlock on your phone; reading handwritten digits (we’ll build this below); detecting tumours in X-rays.
Shape: image → [conv] → [pool] → [conv] → [pool] → [dense] → output

3. Recurrent Neural Network (RNN)

What it is: A network with memory. It processes data one step at a time and passes a hidden state forward, so each step knows what came before. That’s what a feedforward network can’t do.
Best for: Sequences — anything where order matters.
Real example: Predicting the next word in a sentence; forecasting tomorrow’s temperature from the past week; tagging each word in a sentence.
Shape: x₁ → [cell] → x₂ → [cell] → x₃ → [cell] → output (same cell, state carried along)

4. LSTM and GRU

What it is: Improved RNNs. A plain RNN forgets things that happened many steps ago. LSTM (Long Short-Term Memory) and GRU (Gated Recurrent Unit) add “gates” that decide what to remember and what to forget, so they handle long sequences far better.
Best for: Longer sequences — paragraphs, long time series, speech.
Real example: Speech recognition; translating a sentence; predicting stock movement from months of data.
Shape: Same as RNN, but each cell has memory gates inside.

5. Transformer

What it is: The architecture behind GPT, Claude, Gemini and every modern large language model. Instead of reading a sequence step by step, a Transformer looks at the whole sequence at once and uses attention to work out which parts matter to which other parts. That lets it train on enormous data in parallel — which is why LLMs exist.
Best for: Text, and increasingly images, audio and code.
Real example: ChatGPT and Claude; Google Translate; code autocomplete; image generation.
Shape: tokens → [attention + dense] × N → output

6. Autoencoder and GAN (briefly)

Autoencoder: Squeezes data down to a small representation and then rebuilds it. Useful for compression, removing noise, and spotting anomalies (if it can’t rebuild something well, that something is unusual).
GAN (Generative Adversarial Network): Two networks compete — one generates fakes, one tries to catch them — and both improve. This is how early AI image generation worked. Both are more specialised; you’ll meet them after the five above.

Comparison Table

Which One Should I Use? (Decision Guide)

You don’t need to memorise the list — you need to answer one question: what shape is my data?

• Rows and columns (a spreadsheet)? → ANN
• Images or anything on a grid? → CNN
• A sequence where order matters, and it’s short? → RNN
• A sequence, and it’s long? → LSTM / GRU
• Text, or you want state-of-the-art on almost anything? → Transformer

Start with the simplest type that fits your data. A CNN on a spreadsheet is wasted effort; an ANN on images will underperform. Get the type right first, then tune.

Setup

$ pip install tensorflow

This installs TensorFlow with Keras built in (Keras is the friendly API on top of TensorFlow — you’ll write Keras code, TensorFlow does the maths underneath). It’s a large download, around 500 MB. A CPU is fine for everything on this page.

Your First Keras Model — XOR in 15 Lines

On page 1, XOR took about 40 lines of NumPy: manual weights, manual bias, manual backpropagation. Here’s the same network in Keras. Every line maps to something you already built by hand.

import numpy as np
from tensorflow import keras
from tensorflow.keras import layers

keras.utils.set_random_seed(42)

X = np.array([[0,0],[0,1],[1,0],[1,1]], dtype="float32")
y = np.array([[0],[1],[1],[0]],         dtype="float32")

model = keras.Sequential([
    keras.Input(shape=(2,)),
    layers.Dense(8, activation="sigmoid"),   # hidden layer
    layers.Dense(1, activation="sigmoid")    # output layer
])

model.compile(optimizer=keras.optimizers.Adam(0.1),
              loss="binary_crossentropy")

model.fit(X, y, epochs=500, verbose=0)

print(model.predict(X, verbose=0).round(3))

Four predictions, matching XOR: near 0, near 1, near 1, near 0. The network learned — in 15 lines, with no backpropagation code written by you.

About those decimals: Keras initialises weights randomly, and even with set_random_seed(42) the exact values differ a little between TensorFlow versions and hardware. You might see 0.001 and 0.998, or 0.01 and 0.99. That’s normal. What matters is the pattern — first and last near 0, middle two near 1. If instead you see all four stuck around 0.5, the network didn’t learn; jump to Common Mistake 3.

Side-by-Side: NumPy (Page 1) vs Keras

Two things changed besides line count. Keras used binary cross-entropy instead of raw error — a loss designed for yes/no outputs that trains more reliably. And it used Adam instead of plain gradient descent — an optimizer that adapts the step size per weight, which is why 500 epochs here beats 10,000 on page 1. You could have written both by hand; Keras just means you don’t have to.

Step 2 — A Real CNN: Handwritten Digits in 25 Lines

Now the type that made deep learning famous. MNIST is 70,000 images of handwritten digits (0–9), each 28×28 pixels. It’s built into Keras — it downloads about 11 MB the first time you run this, then it’s cached. We’ll build a small CNN and get around 99% accuracy in a few minutes on a laptop CPU.

from tensorflow import keras
from tensorflow.keras import layers

keras.utils.set_random_seed(42)

# 1. Load and prepare the data
(X_train, y_train), (X_test, y_test) = keras.datasets.mnist.load_data()
X_train = X_train.reshape(-1, 28, 28, 1) / 255.0   # add channel, scale 0–1
X_test  = X_test.reshape(-1, 28, 28, 1)  / 255.0

# 2. Build the CNN
model = keras.Sequential([
    keras.Input(shape=(28, 28, 1)),
    layers.Conv2D(32, 3, activation="relu"),   # detect edges
    layers.MaxPooling2D(),                     # shrink
    layers.Conv2D(64, 3, activation="relu"),   # detect shapes
    layers.MaxPooling2D(),
    layers.Flatten(),                          # grid → vector
    layers.Dense(64, activation="relu"),
    layers.Dense(10, activation="softmax")     # 10 digits
])

# 3. Train
model.compile(optimizer="adam",
              loss="sparse_categorical_crossentropy",
              metrics=["accuracy"])
model.fit(X_train, y_train, epochs=3, batch_size=128,
          validation_split=0.1)

# 4. Test on images it has never seen
loss, acc = model.evaluate(X_test, y_test, verbose=0)
print(f"Test accuracy: {acc:.4f}")

Three epochs, under a minute on a laptop CPU, and the network reads handwritten digits it has never seen with 98.7% accuracy. Read the shape of the code: two Conv2D + MaxPooling2D pairs extract features, Flatten turns the 2D grid into a vector, and two Dense layers classify — that’s the CNN pattern from the diagram, and it’s the same pattern used (at larger scale) in real image systems.

Three choices are worth noticing. The output layer has 10 neurons with softmax — one per digit, and softmax makes them sum to 1 so the biggest one is the prediction. The loss is sparse_categorical_crossentropy — “sparse” because our labels are plain integers (0–9), not one-hot vectors. And validation_split=0.1 holds back 10% of the training data to check progress — the test set stays untouched until the very end, which is the train/test discipline from the Data Science notes.

Common Mistakes (and the Exact Error You’ll See)

Mistake 1: Wrong input shape

ValueError: Input 0 of layer "sequential" is incompatible with the layer:
expected shape=(None, 28, 28, 1), found shape=(None, 28, 28)

Cause: You skipped the reshape. MNIST loads as (28, 28) but Conv2D needs a channel dimension — (28, 28, 1) for greyscale, (28, 28, 3) for colour.
Fix: X_train = X_train.reshape(-1, 28, 28, 1). The -1 means “work out the number of images for me.”

Mistake 2: “expected min_ndim=2, found ndim=1”

ValueError: Exception encountered when calling Sequential...
Input 0 of layer "dense" is incompatible with the layer:
expected min_ndim=2, found ndim=1

Cause: You predicted on a single sample like model.predict(np.array([1, 0])). Keras always expects a batch — a 2D array of samples, even if there’s only one.
Fix: Add the batch dimension: model.predict(np.array([[1, 0]])) — note the double brackets. Same idea as X = df[["hours"]] in the Data Science notes.

Mistake 3: Loss not decreasing (wrong activation / loss combination)

Cause: The output activation and the loss must match the problem. The three combinations that work:

Mix them up — sigmoid with 10 classes, or categorical_crossentropy with integer labels — and you’ll either get an error like Shapes (None, 1) and (None, 10) are incompatible, or worse, the loss just sits still. If XOR gives you four 0.5s, this table is the first place to look. Second place: learning rate — try 0.1 for tiny problems, 0.001 (Adam’s default) for real ones.

Mistake 4: Training takes forever on CPU

Cause: Too many epochs, a batch size that’s too small, or a model bigger than the problem needs.
Fix: For MNIST, epochs=3, batch_size=128 is plenty — don’t start with 50 epochs. If you genuinely need more compute, Google Colab gives you a free GPU: Runtime → Change runtime type → GPU. Your code runs unchanged.

Experiments — Try These Yourself

Change one thing, re-run, watch what happens. These are typical results — yours will vary a little, and that’s fine.

Practice Exercises

• AND gate in Keras. Change y to [[0],[0],[0],[1]] and re-run the XOR model. It should learn in far fewer epochs — try epochs=100. Why is AND easier than XOR? (Hint: can you draw a single straight line that separates the 1 from the 0s?)
• Look at what the CNN gets wrong. After training, run preds = model.predict(X_test).argmax(axis=1) and find indices where preds != y_test. Plot a few with plt.imshow(X_test[i].reshape(28,28), cmap="gray"). Are they genuinely ambiguous digits?
• Count the parameters. Run model.summary() on both models. The XOR model has 33 trainable parameters — can you account for each one from the layer sizes? (8 hidden × 2 inputs + 8 biases + 8 × 1 + 1 bias.) Then look at the CNN’s count and find which layer has the most.

Where to Go Next

You now know the types and you’ve trained two of them. The natural next steps:

• Sequences: swap Conv2D for layers.LSTM and try a simple time-series forecast — the Keras pattern is identical.
• Real projects: the Data Science with Python from scratch notes cover the cleaning and evaluation discipline you’ll need before any model, Keras or otherwise.
• The bigger picture: where Transformers and LLMs fit in the field — see the AI from scratch for beginners notes.
• Transformers hands-on: once the above is comfortable, the Hugging Face transformers library lets you run a real pre-trained model in about 5 lines.

📥 Download These Notes as PDF

Want to study offline? Download the complete Types of Neural Networks notes — all six types with examples, the comparison table and decision guide, both Keras models with their code and output, the four common mistakes and the experiments table — as a free PDF from TechProFree. No signup required.

Final Thoughts

The family of neural networks looks big until you see the pattern: they’re all layers, weights, bias and activation — the same four parts from page 1 — wired differently for different data. Tabular → ANN. Images → CNN. Sequences → RNN/LSTM. Text → Transformer. Pick by data shape, start simple, tune after.

And you’ve now seen both sides of Keras: it took your 40 lines of NumPy down to 15, and it took a real image-recognition network down to 25. Run both, do the experiments, break them on purpose with the mistakes above — that’s how the types stop being a list and start being tools you reach for.

Which type are you going to try next — LSTM on a time series, or a bigger CNN? Hit a different error than the four above? Drop a comment — I read and reply to every one.

Frequently Asked Questions (FAQ)