Pruning and Optimizing Large Language Models in an Era of GPU Scarcity

The increasing computational and environmental costs associated with AI models, especially large language models (LLMs), highlight the urgent need for network optimization. These models consume vast amounts of energy and resources, requiring more efficient training strategies to balance performance with ecological responsibility. Our focus is on enhancing the efficiency of deep neural networks on embedded devices through novel pruning techniques: “evolution of weights” and “smart pruning.” These methods, compared to traditional pruning approaches using benchmark datasets, involve evaluating parameter importance during training to better preserve accuracy during compression. Our approach results in faster computations and higher compression rates with minimal accuracy loss. We have successfully applied these techniques to LLMs consisting of around 10 million parameters. The LLM experiment is publicly available on Github to facilitate replication testing.