Advertisement
Old 4 Feb 2010, 10:39 PM EST #1
ian ryan
forum dweller

ian ryan's Avatar

join date: Apr 2007
Posts: 9,000
ian ryan is on a distinguished road

Capstone - Classification Convolutional Neural Network by Ian Ryan

https://github.com/ianhenryryan/capstone

Welcome to My Fall 2024 Capstone Project advised by Dr. Soltys.

The objective of this semester-long project was to create a Convolutional Neural Network (CNN) using PyTorch by scratch, rather than using pre-trained models like ResNet-18. After developing an acceptable classification model, the next goal was to adapt it into an object detection model by implementing YOLOv8.

Although my primary interest was detecting humans, I chose to create a multiclass model to avoid building a simple perceptron. To up the ante, I also made the dataset that the model was trained on an imbalanced dataset. Developing my own model presented a challenging learning opportunity and helped strengthen my understanding of CNN fundamentals for computer vision, a field in which I aspire to build a career.

This repository contains the majority of my work, including various iterations of fine-tuning, visualizations, the CNN architecture schematic, datasets used, comprehensive Jupyter Notebooks, my capstone poster, final capstone pitch, literature citations, and more. The Jupyter Notebooks feature detailed indexes for easy navigation through different sections of the programs.

Convolutional Neural Network Architecture

IR Architecture

Notebook Index

Table of Contents

  • Libraries & Imports .................................... 1
    • Libraries ........................................ 2
    • Imports ......................................... 3
  • CUDA GPU Availability ............................... 4
  • Download & Load Datasets ............................ 5
    • First Dataset Resource .......................... 6
    • Second Dataset Resource ......................... 7
  • Functions .......................................... 8
    • Path Datasets .................................. 9
    • First Dataset .................................. 10
    • Second Dataset ................................. 11
  • Pre-Processing The Datasets ........................ 12
    • Combine Datasets ............................... 13
    • Save Combo Set ................................. 14
    • Redistribute Images Dynamically Function ....... 15
  • Pathing (Jump here after Libraries & Imports if using own dataset and not combining into imbalanced set.) 16
    • Normalize File Extensions ...................... 17
    • Extensive Pathing Checking ..................... 18
  • Transforms & Loaders .............................. 19
  • Classification Or Object Detection Model? ......... 20
    • Classification Task ............................ 21
    • Regression Task ................................ 22
  • CNN Model ........................................ 23
    • Define Class Weights .......................... 24
    • Define CNN Model Architecture ................. 25
    • Weights & Biases .............................. 26
    • Weight Initialization ......................... 27
    • Hyperparameters ............................... 28
  • Class Confirmation in Training Data .............. 29
  • Profile Memory Usage ............................. 30
  • Training Prep .................................... 31
  • Training CNN Model ............................... 32
  • Test Accuracy of Convolutional Neural Network ... 33
  • Visualizations .................................. 34
    • Summary - Everything .......................... 35
    • Model Performance Function .................... 36
    • Training & Validation Loss Plots .............. 37
    • Confusion Matrices ............................ 38
    • Feature Maps ................................. 39
    • Kernel Visualizations ........................ 40
    • Gradient Visualizations ...................... 41
    • CAM / Grad-CAM ............................... 42
    • Training/Validation Curves ................... 43
    • Explainability Tools ......................... 44
  • Acknowledgements ................................ 45
  • Literature Cited ................................ 46
  • Environment .................................... 47
  • Recommended Resources .......................... 48
  • Creator Information ............................ 49

Acknowledgements

I would like to express my gratitude to Dr. Michael Soltys and Dr. William Barber for sparking my interest in Machine Learning and Artificial Intelligence. Over the past two years at California State University Channel Islands (CSUCI), it has been a pleasure taking multiple courses with both professors, each bringing their own perspectives and experiences. They both were able to articulate complex information in a digestible way effortlessly.

Dr. William Barber
Dr. Barber, with his background in Physics and a distinguished career in Medical Imaging and Research, provided a scientific and mathematical foundation that enhanced my understanding of computational models, data analysis, and data extraction. His extensive industry experience includes serving as a Director of Medical Imaging. His engaging teaching style and passion for image processing techniques and pattern recognition concepts inspired me to explore this field further.

Dr. Michael Soltys
Dr. Soltys, with his comprehensive background in Algorithms, Machine Learning, and Cloud Computing, fostered my interest in pursuing a Computer Vision Capstone Project. As an accomplished author of two books and more than 60 published research papers, and as a Principal Scientist and Software Engineer, Dr. Soltys brings a wealth of in-depth knowledge and real-world applications to his teaching. His courses and recommendations proved to be instrumental during the development of my Convolutional Neural Network (CNN) Capstone Project.

Bayne H. Ryan
Bayne, my dear nephew. You arrived the first week of December, right in the middle of preparations for my Capstone Showcase. I truly admire your impeccable sense of urgency to stop being a submarine and surface just in time to appreciate my Convolutional Neural Network (CNN) Capstone project. Your surprise guest appearance was undoubtedly the highlight of my year. I cannot wait to forcibly teach you calculus when you reach the prestigious age of juice boxes and nap time (also known as four years old).

Literature Cited

Tan, M., & Le, Q. V. (2019). EfficientNet: Rethinking Model Scaling for Convolutional Neural Networks. arXiv preprint arXiv:1901.08688.

Brownlee, J. How to Develop Convolutional Neural Network Models for Time Series Forecasting. Machine Learning Mastery. Available at: https://machinelearningmastery.com/how-to-develop-convolutional-neural-network-models-for-time-series-forecasting/.

Dougherty, G. Pattern Recognition and Classification: An Introduction. Springer, 2012.

Eliot, D. Deep Learning with PyTorch Step-by-Step: A Beginner's Guide, Volume I: Fundamentals. Self-published, 2020.

Jocher, G., Chaurasia, A., & Qiu, J. (2023). Ultralytics YOLOv8. Available at: https://github.com/ultralytics/ultralytics.

Environment

AWS SageMaker Jupyter Notebook

  • Notebook Instance Type: ml.g4dn.2xlarge
  • Processor: NVIDIA T4 Tensor Core GPU with 16 GB GDDR6 VRAM
  • Memory: 32 GB System RAM
  • Storage: 5 GB EBS Volume
  • Operating System: Amazon Linux 2 with Jupyter Lab 3 (notebook-al2-v2)
  • Lifecycle Configuration: None
  • Elastic Inference: Not Enabled
  • Minimum IMDS Version: 2

    • Recommended Resources

    Courses for Computer Science Students at California State University Channel Islands (CSUCI) Interested in Machine Learning & AI

    Channel Island Specific Courses:
    • COMP 345 - Digital Image Processing (Fall Course)
    • COMP 354 - Analysis of Algorithms
    • COMP 445 - Image Analysis and Pattern Recognition (Spring Course)
    • COMP 454 - Automata, Languages & Computation
    • COMP 469 - Intro to Artificial Intelligence / Machine Learning (Fall Course)
    • COMP 491 - Capstone Prep (Specifically with Dr. Soltys)
    • COMP 499 - Capstone (Specifically with Dr. Soltys)
    LinkedIn Learning Courses (if offered):
    If a College/University student, it is worth seeing what resources your institution provides at your disposal. For example, CSUCI offers LinkedIn Learning and access to the O'Reilly website.
    • Deep Learning and Generative AI: Data Prep, Analysis, and Visualization with Python - Leverage Generative AI for Analytics and Insights by Gwendolyn Stripling
    • Deep Learning: Image Recognition - Learning Image Recognition by Isil Berkun
    • Advanced AI: Transformers for Computer Vision by Jonathan Fernandes
    • Building Computer Vision Applications with Python by Eduardo Corpeño
    • Applied Machine Learning: Algorithms by Matt Harrison
    • Building Deep Learning Applications with Keras by Isil Berkun
    Helpful Resources:
    • Think Python: How to Think Like a Computer Scientist (2nd Edition, Version 2.4.0) by Allen Downey
    • Deep Learning with PyTorch Step-by-Step: A Beginner's Guide - Volume I: Fundamentals by Daniel Voigt Godoy
    • Pattern Recognition and Classification: An Introduction (Springer, 2012) by Geoff Dougherty
    • An Introduction to the Analysis of Algorithms: (3rd Edition, 2018) by Michael Soltys
    • Ultralytics YOLOv8 Documentation
    • Object Detection and Localization with YOLOv3 by B. Rupadevi and J. Pallavi
    • Digital Image Processing for Medical Applications by Geoff Dougherty

    Creator Information

    Permission

    Students, educators, and anyone else keen on learning about Convolutional Neural Networks (CNNs), Computer Vision, Supervised Learning Algorithms, Machine Learning, Deep Learning, AI, or related fields are welcome to use this notebook in any capacity as a learning resource.

    This notebook is part of my Fall 2024 Capstone Project for my Bachelor’s degree in Computer Science at California State University Channel Islands. I structured the content to be approachable and digestible, reflecting my own learning journey in AI and Machine Learning.

    I hope this notebook can be of use to those exploring similar topics. This specific Jupyter Notebook focuses on Image Classification and demonstrates combining two datasets to create a class imbalance for training purposes. A separate notebook dedicated to Object Detection will be available soon (if not already).

    Important Note on Datasets: The datasets used in this project are not my property. Credit is given to the original dataset creators, and their links are provided within the notebook in the dataset section at the beginning.

    I understand that grasping the fundamentals of CNNs and related AI concepts can be overwhelming at first. My goal is to make these topics more accessible through these notebooks.

    Permissions:
    You are free to download, use, edit, and reference the notebooks, Python code, and Markdown content. I aim for accuracy in the explanations provided, though I acknowledge that scientific understanding is always evolving. I welcome constructive feedback and corrections.

    This project is intended as a learning resource.

    Ian Ryan

    Grá Mór
    ian ryan is offline Reply With Quote
    Unread 4 Feb 2010, 11:24 PM EST #2
    ian ryan
    forum dweller

    ian ryan's Avatar

    join date: Apr 2007
    posts: 9,000
    ian ryan has a spectacular aura about

    Deconvolution (De-blur) & Enhancement by Ian Ryan


    A deep dive into digital image processing using ImageJ to correct blurry images through deconvolution and enhance poorly illuminated images. Techniques used include Wiener filtering, custom PSFs, and enhancement via brightness/contrast normalization, often layered with macros to test results efficiently.

    Table of Contents

    • Introduction
    • Methods
    • Deconvolution Results
      • blurred.tif (License Plate)
      • brainGB2.tif (Skull)
      • carnivalride.png (Rotational Blur)
      • vintagephotographer.tif
    • Enhancement Results
      • bad_background.jpg
      • dark_foreground.jpg
    • Conclusion
    • Resources & Environment
    • Downloadable Macros

    Introduction

    The project focused on sharpening blurry images and enhancing dark ones using various digital image processing techniques. This was done in ImageJ, with careful experimentation on gamma values, deconvolution filters, and visual comparison of results.

    Deconvolution (De-Blur) initial images Seed Class Example

    Enhancement initial images
    Seed Class Example

    Methods

    • Software
      • ImageJ (Fiji distribution)
    • Filters
      • Wiener Filter (Deconvolution)
      • Enhance Contrast
      • Sharpen
      • Despeckle
      • Brightness/Contrast
      • Shadow Enhancement
    • Point Spread Functions (PSF)
      • pillbox.tif
      • GaussianBlur.tif (with custom modifications)
      • Handcrafted disc-shaped PSF (for carnivalride.png)
    • Custom Macros
      • One macro per image with step-by-step automation
      • Used for documentation and reproducibility
    • Hardware
      • Alienware M15 Nvidia 3060
      • Windows 11 with ImageJ

    Deconvolution Results

    • blurred.tif (License Plate)
      • Used: Wiener Filter (gamma = 0.05), pillbox PSF
      • Enhancements: Sharpen → Shadows (SW) → Invert
      • Goal: Make the plate readable
      • Before / After: Seed Class Example
    • brainGB2.tif (Skull)
      • Custom PSF based on modified Gaussian blur
      • Steps: Wiener Filter (0.005) → Sharpen → Enhance Contrast
      • Also explored a non-convolution deblur trick using subtraction
      • Before / After: Seed Class Example
    • carnivalride.png (Rotational Blur)
      • Most difficult case (rotational motion blur)
      • Hand-drawn disc-shaped PSF used
      • Multiple iterations of deconvolution and blending
      • Before / After: Seed Class Example
    • vintagephotographer.tif
      • Modified Gaussian PSF
      • Complex multi-step process involving mean filter, math gamma, image subtraction, despeckle, etc.
      • Before / After: Seed Class Example
  • Custom Made Point Spread Functions for Project.
    • Seed Class Example

    • Enhancement Results

    • bad_background.jpg
      • Brightness adjusted -> Enhance contrast
      • Result: Partial face revealed
      • Before / After: Seed Class Example
    • dark_foreground.jpg
      • MATH gamma failed, fallback to brightness + histogram equalization
      • Result: Visible facial details without losing the background
      • Before / After: Seed Class Example

    Conclusion

    The project was a rewarding challenge, I despised dealing with rotational blur in the carnival ride. Experimenting with the Wiener filter, custom PSFs, and enhancement orders provided valuable insight into digital image restoration.

    Environment

    • Software
      • ImageJ (Fiji distribution)
    • System
      • Alienware m15 Nvidia GeForce RTX 3060
      • Windows 11
    • File Types
      • .tif
      • .jpg
      • .png

    Macros

    Capable of sending macro zip upon request if interested.


    Grá Mór
    daddy is offline Reply With Quote
    Unread 5 Feb 2010, 12:15 AM EST #3
    ian ryan
    forum dweller

    ian ryan's Avatar

    join date: Apr 2007
    Posts: 9,000
    Why.So.Saucey is a glorious beacon of light
    Send a message via AIM to daddy

    Seed Pattern Recognition by Ian Ryan

    This project explores seed classification using pattern recognition techniques. I used self-captured seed images, processed them in ImageJ to extract measurable features, and then trained a K-Means Clustering model in JMP. The goal was to accurately differentiate between 5 seed types based on shape-based features.

    Table of Contents

    • Seed Classes
    • Image Capturing
    • Image Processing + Segmentation
      • blurred.tif (License Plate)
      • brainGB2.tif (Skull)
      • carnivalride.png (Rotational Blur)
      • vintagephotographer.tif
    • Classification Models
      • K-Means Clustering
      • Neural Net
    • Visuals
    • Conclusion

    Seed Classes

    This project contains 5 classes: Weed, Carrot, Pumpkin, Pea, Spinach
    Training Data: 5 classes - 4 images per class
    Testing Data: 5 classes - 3 images per class Seed Class Example

    Image Capturing

    This project focuses on classifying a variety of seeds. Conveniently, my parents grow vegetables and my dad reliving his glory days from the 1970s, grows weed. This left a plethora of seeds to choose from to capture images of and then extract features from. The chosen classes of seed types utilized are weed, carrot, pumpkin, pea, and spinach.
    Seed Class Example first raw training image of each class.

    For capturing the images I placed a piece of 8.5x11 white paper into an In-N-Out tray. After set-up of the environment I poured seeds into the tray and captured 4 images per seed type to gather the training data using an Iphone 13. I repeated this process again and captured 3 images per seed type to gather testing data since you cannot test with the data you trained on. All images captured at various distances, meaning area cannot be used as a feature for the classes.

    Image Processing + Segmentation + Feature Extraction

    For the processing of the images in both the training and testing images I utilized a standard approach to all of them using the same techniques and made minor adjustments to the techniques as needed. Over a few hours of tinkering in ImageJ with the images and extracting their data, I successfully segmented and feature extracted both sets of images.

    Techniques Performed
    • 8-bit
    • Duplicate
    • Subtract Background
    • Enhance Contrast
    • Duplicate
    • Gaussian Blur
    • ImageCalc: Duplicate of Duplicate subtract Duplicate
    • Enhance Contrast
    • Threshold
    • Remove Outliers
    • Fill Holes
    • Set Measurements
    • Analyze Particles
    • Invert LUT
    Train Weed before/after:
    Seed Class Example Seed Class Example

    Train Carrot before/after:
    Seed Class Example Seed Class Example

    Train Pumpkin before/after:
    Seed Class Example Seed Class Example

    Train Pea before/after:
    Seed Class Example Seed Class Example

    Train Spinach before/after:
    Seed Class Example Seed Class Example

    Classification Models

    Now for putting the extracted training data to use. Following the image processing and segmentation we now have acceptable data/features/measures to train the model with. The next step is to export all the data from Google Sheets to an XLS file. Once that is done I open the statistical software JMP (student edition). Here I click on New Data Table which brings up an empty array table. Then press open and click on the all XLS file created earlier. Import it and then save it as a JMP file. Seed Class Example

    K-Means Clustering

    • Used JMP Statistical Software (Student Version)
    • Imported all labeled feature data into a data table
    • Set number of clusters = 5 (one per seed class)
    • Results:
      • Decent clustering overall
      • Noticeable overlap between certain seed types
      • Visually similar seeds (like carrot and pea) contributed to the overlap
    Seed Class Example Here is the scatterplot matrix, As you can see there is a lot of overlap and overlapping in the clusters. Seed Class Example

    Neural Net (out of curiousity, not enough data for accurate NN)

    Seed Class Example Seed Class Example

    Conclusion

    In conclusion, the segmentation portion of the project took me the longest to complete. There were cases in which I had to keep trying to process images over and over again until I produced an acceptable number of objects in each image. It is important to crop the testing and training images so there is less noise to try and eliminate from the images. It is also a good idea to overly take measurements/data to give more options on how to manipulate it in certain ways to benefit the model more. Using seeds that are more varying in circularity and shape would have been beneficial as a lot of my data gets misread in the models.

    Specifications

    • Camera: iPhone 13
    • Alienware m15 R7 Laptop
    • Processor:
      • AMD Ryzen 7 6800H (20 MB total cache, 8 cores, 16 threads)
    • Graphics Card:
      • NVIDIA GeForce RTX 3060, 6GB GDDR6
    • Memory:
      • 16 GB (2 x 8 GB, DDR5, 4800 MHz)
    • Hard Drive:
      • Crucial T500 2TB PCIe Gen4 NVMe M.2 SSD
    • Operating Systems:
      • Windows 11
    • Software:
      • ImageJ
      • Google Sheets
      • JMP: Statistical Software: Student Version

    Resources

    • Images:
      • Training Images:
        • 4 x Images (self-taken) x 5 different classes (seed types)
      • Testing Images:
        • 3 x Images (self-taken) x 5 different classes (seed types)
    • Code:
      • ImageJ Macros:
        • 4 x Segmentation Training Macros x 5 different classes XLS files
        • 3 x Segmentation Testing Macros x 5 different classes XLS files
    • Data:
      • Training Data:
        • 4 x Classes (images) x 5 different classes (seed types) XLS files
        • 1 x All Training Data XLS file with label
      • Testing Data:
        • 3 x Classes (images) x 5 different classes (seed types) XLS files
        • 1 x All Testing Data XLS file with label


    Please reach out if interested in the project zip file containing all resources.

    Grá Mór
    daddy is offline Reply With Quote
    Reply

    Previous Thread | Next Thread

    Thread Tools
    Display Modes
    Linear Mode Linear Mode

    Posting Rules
    You may not post new threads
    You may not post replies
    You may not post attachments
    You may not edit your posts
    vB code is On
    Smilies are On
    [IMG] code is On
    HTML code is Off

    All times are GMT -4. The time now is 4:39 AM EST.
    Powered by: vBulletin, Copyright \A92000 - 2010, Jelsoft Enterprises Ltd.