AI academy - Small Data, Big Vision: Reusing Powerful Models in Low-Resource Settings

This workshop focuses on the practical question of how to obtain strong visual models when the available dataset is too small to support conventional deep learning from scratch.

The keynote introduces transfer learning as a strategy for reusing representations learned on large image collections. Participants examine why early and intermediate CNN features can transfer across tasks, how feature extraction differs from fine-tuning, and why learning rate, layer freezing, augmentation, and domain similarity matter when adapting a pretrained network.

The case study is framed around a low-resource recognition problem with only a limited number of labelled images per class. Participants first establish a small-data baseline, then reuse a pretrained CNN such as MobileNet, EfficientNet, or ResNet. They compare frozen-feature extraction with selective fine-tuning and analyse the trade-offs among performance, training time, data requirements, and computational cost.

In the hands-on lab, participants create a transfer-learning pipeline in Keras/TensorFlow, add a task-specific classification head, tune augmentation, unfreeze selected layers, and compare model behaviour under progressively smaller training subsets. They also consider class imbalance and domain shift as practical constraints.

Morning

Keynote Lecture: Why Start from Scratch?

Topics covered through the keynote

• Pretrained CNNs and reusable visual representations
• Feature extraction versus fine-tuning
• Layer freezing, learning rates, and catastrophic forgetting
• Data augmentation for small datasets
• Domain similarity and domain shift
• Class imbalance and validation under limited samples
• Accuracy, model size, training cost, and deployment trade-offs

Hands-on Lab: Transfer Learning Under a Data Budget

• Build a baseline model from scratch
• Load and adapt a pretrained vision backbone
• Compare frozen feature extraction with selective fine-tuning
• Evaluate performance on progressively smaller training subsets
• Measure overfitting, training time, and model complexity
• 
  

Afternoon

Group Case Study: Design a Vision System with Limited Data and Compute

Participants work in groups under a fixed resource budget and define:

• the pretrained architecture and rationale for selection
• the freezing and fine-tuning strategy
• augmentation and class-balancing choices
• evaluation criteria and stopping rules
• risks related to domain shift and limited evidence
• 
  

Group Presentations and Trade-off Review

Short presentations (approximately 5 minutes per group)
Discussion of performance, cost, generalisation, and deployment constraints

By the end of the course, participants will be able to:

• explain why pretrained visual representations can transfer to new tasks
• distinguish feature extraction from fine-tuning
• adapt a pretrained CNN to a new image-classification problem
• apply augmentation and layer-freezing strategies for small datasets
• compare performance and cost across alternative transfer-learning configurations
• recognise risks related to overfitting and domain shift
• design an evidence-based transfer-learning strategy under resource constraints
• 
  

One-day intensive workshop combining:

• case-study keynote lecture
• hands-on transfer learning in Jupyter notebooks
• controlled small-data experiments
• group-based system design under resource constraints
• presentations and trade-off discussion