Reducing NLP Model Embeddings for Deployment in Embedded Systems

State-of-the-art natural language processing (NLP) models have revolutionized the way machines understand, generate, and summarize human language; however, these modern techniques take advantage of the general abundance of available computing resources. Deploying such models into resource restricted and/or embedded systems is severely limited due to their memory, network, and power demands. When these models are deployed in resource-limited environments, users must determine the maximum performance degradation they are willing to withstand to meet the requirements of the implementation. This study builds on prior research that assessed the effectiveness of smaller BERT models for use in resource-limited settings. It evaluates the performance of reduced-size BERT models in named-entity recognition (NER) tasks. The main focus is on investigating whether reducing the token embedding size of a model through various dimension-reduction methods can maintain a tolerable level of performance while enabling deployment to more restricted compute environments. In particular, this study employs principal components analysis (PCA), truncated singular value decomposition (TSVD), agglomerative clustering (AC), and uniform manifold approximation and projection (UMAP) to reduce the embedding matrix of pre-trained DistilBERT and discuss optimal hyperparameters.