Researchers at UC Santa Cruz have developed a method to significantly reduce the energy required to run large language models.
This development could significantly impact the use of artificial intelligence (AI) in e-commerce. By reducing power consumption, their approach could make advanced AI capabilities more accessible and affordable for businesses of all sizes.
“We achieved the same performance at a much lower cost – we just had to fundamentally change the way neural networks work,” said Jason Eshraghian, assistant professor of electrical and computer engineering at UC Santa Cruz’s Baskin School of Engineering and lead author of the study, in a press release Thursday (June 20). “Then we went a step further and built custom hardware.”
The costs of AI in e-commerce
Currently, running advanced AI models like ChatGPT comes at a high cost. Recent estimates put OpenAI’s energy costs alone at nearly $700,000 per day. This cost is rolled into the price and could pose a significant hurdle for smaller companies looking to use AI in their e-commerce operations.
The UC Santa Cruz team’s research aims to reduce the high energy costs associated with running advanced AI models. By eliminating matrix multiplication, the most computationally intensive element of running large language models, they were able to make the model more energy efficient.
“Neural networks are, in some ways, glorified matrix multiplication machines,” Eshraghian said. “The bigger your matrix, the more things your neural network can learn.”
The researchers claim that their approach is remarkably efficient.
“We were able to run a language model with a billion parameters using only 13 watts. That’s about the energy needed to run a lightbulb and is more than 50 times more efficient than conventional hardware,” said Eshraghian.
This level of efficiency could enable e-commerce platforms to offer advanced AI-driven features such as personalized recommendations, chatbots and dynamic pricing at a fraction of current costs.
Impact on mobile e-commerce
The team’s innovation also has significant implications for mobile e-commerce. Rui-Jie Zhu, lead author of the study and a doctoral student in Eshraghian’s group, noted in the press release: “We replaced the expensive surgery with cheaper ones.”
The reduction in computational complexity achieved by the UC Santa Cruz team could potentially enable comprehensive AI models to run on smartphones. This advancement comes at a time when mobile shopping is growing rapidly.
If implemented, this technology could significantly improve the mobile shopping experience and app-based e-commerce by enabling more sophisticated AI-powered features such as personalized recommendations and advanced search capabilities to run directly on users’ devices.
Building on their software advances, the team expanded their research by collaborating with other faculty members at UC Santa Cruz to develop custom hardware. This specialized hardware was designed to maximize the efficiency gains of their new approach.
“These numbers are already very solid, but it’s very easy to make them much better,” said Eshraghian. “If we can do that with 13 watts, just imagine what we could do with the processing power of an entire data center. We have all these resources, but let’s use them effectively.”
For e-commerce giants with massive data centers, this could mean significant cost savings and improved AI capabilities. For smaller companies, it could level the playing field and allow them to compete with more sophisticated AI-driven strategies.
As PYMNTS previously reported, major technology companies like Microsoft and Google are struggling to make profits from their generative AI-based products due to high production, development and training costs.
As the e-commerce industry continues to evolve, innovations like these could transform the way businesses interact with customers, manage inventory, and make strategic decisions. Although the technology is still in its early stages, its potential to democratize advanced AI capabilities in the e-commerce sector is enormous.
The researchers have released their model as open source, potentially accelerating adoption and further innovation in the field. As Eshraghian puts it, “We have fundamentally changed the way neural networks work.” The e-commerce world will be watching closely to see how this shift translates into real-world applications and competitive advantages in the digital marketplace.
