One of the largest factors affecting disease recurrence after surgical cancer resection is negative surgical margins. Hyperspectral imaging (HSI) is an optical imaging technique with potential to serve as a computer aided diagnostic tool for identifying cancer in gross ex-vivo specimens. We developed a tissue classifier using three distinct convolutional neural network (CNN) architectures on HSI data to investigate the ability to classify the cancer margins from ex-vivo human surgical specimens, collected from 20 patients undergoing surgical cancer resection as a preliminary validation group. A new approach for generating the HSI ground truth using a registered histological cancer margin is applied in order to create a validation dataset. The CNN-based method classifies the tumor-normal margin of squamous cell carcinoma (SCCa) versus normal oral tissue with an area under the curve (AUC) of 0.86 for inter-patient validation, performing with 81% accuracy, 84% sensitivity, and 77% specificity. Thyroid carcinoma cancer-normal margins are classified with an AUC of 0.94 for inter-patient validation, performing with 90% accuracy, 91% sensitivity, and 88% specificity. Our preliminary results on a limited patient dataset demonstrate the predictive ability of HSI-based cancer margin detection, which warrants further investigation with more patient data and additional processing techniques to optimize the proposed deep learning method.