In the food and medical packaging industries, clean packaging is crucial to both customer satisfaction and hygiene. An operational Quality Assurance Department (QAD) is necessary for detecting contaminated packages. Manual examination becomes tedious and may lead to instances of contamination being missed along the production line. To address this issue, a system for contamination detection is proposed using an enhanced deep convolutional neural network (CNN) in a human–robot collaboration framework. The proposed system utilizes a CNN to identify and classify the presence of contaminants on product surfaces. A dataset is generated, and augmentation methods are applied to the dataset for nine classes such as coffee, spot, chocolate, tomato paste, jam, cream, conditioner, shaving cream, and toothpaste contaminants. The experiment was conducted using a mechatronic platform with a camera for contamination detection and a time-of-flight sensor for safe machine–environment interaction. The results of the experiment indicate that the reported system can accurately identify contamination with 99.74% mean average precision (mAP).