No Need to Add More GPUs, Yet Throughput Up to 95% Higher… Nota Inc. Supplies AI Coding Agents to the Financial Sector
Dube and Financial Sector Partner on On-Premises AI Business
Comprehensive Solutions Ranging from LLM Lightweighting, Inference, and Serving to Servers
Wait Time for First Response Cut by More Than Half
Protecting Source Code and Business Information on a Closed Network Without External APIs
[Edaily Reporter Lee So-Hyun ] #Nota Inc., a company specializing in optimizing AI inference, is setting its sights on the artificial intelligence (AI) coding agent market, targeting use even within the financial sector’s closed networks. By optimizing large language models (LLMs), the company has increased the number of requests processed per minute by up to 95% in the same GPU environment and reduced the wait time for the first response to less than half.
Nota Inc. has signed a contract with financial IT firm Dube to supply optimized large language models (LLMs) and servers for AI coding agents tailored to the financial sector. (Photo: Nota Inc.)
Nota Inc. announced on the 16th that it has signed a contract with financial IT firm Dube to supply optimized LLMs and servers for a financial sector-specific on-premises Agent AI business and has completed the delivery.
Nota Inc. provides an integrated solution tailored to Dube’s AI coding agent, including model lightweighting, optimization of the inference and serving environments, and server configuration.
As a result of these optimizations, the number of requests processed per minute in the same GPU environment increased by approximately 58–95% compared to before implementation, and the time from requesting a response to generating the first token was reduced to less than half.
The company explained that since the system can process requests from more concurrent users on the same hardware, businesses can reduce the burden of having to deploy additional GPUs as the number of Agent AI users grows.
Running LLMs on Internal Servers Without External APIs
This initiative is aimed at financial institutions where security regulations restrict the use of cloud-based generative AI.
In the financial sector’s development environment, it is difficult to transmit source code and internal business information externally. The AI coding agent provided by Nota Inc. and Dubé operates on-premises, running the LLM on the client’s internal servers.
Since it does not call external AI APIs, financial institutions can utilize AI without sending source code or internal development information outside the organization.
Dube’s AI coding agent supports development-related tasks, ranging from analyzing and organizing development requirements to developing screens and interfaces. It can also incorporate each financial institution’s coding rules and internal libraries.
Enhanced GPU Efficiency Even with Tens of Thousands of Token Requests
Nota Inc. has applied optimization technologies tailored to the nature of coding agents, which must process large volumes of code and development guidelines in a single request.
As requests containing tens of thousands of tokens are repeated and the number of concurrent users increases, GPU memory usage can rise and response speeds may slow down.
Nota Inc. has used quantization technology to reduce the LLM’s memory usage while simultaneously optimizing both the inference process for generating responses and the serving environment for handling multiple requests concurrently.
Building on this agreement, Nota Inc. and Dube plan to jointly propose the adoption of on-premises Agent AI tools to clients in the financial sector. They will also expand the supply of optimized LLMs and servers tailored to each client’s user base, security, and operational environment.
Nota Inc. plans to expand its AI inference and serving optimization business beyond the financial sector to other industries where the use of external cloud services is restricted.
Chae Myeong-su, CEO of Nota Inc., stated, “As the enterprise Agent AI market expands, not only model performance but also the ability to stably process a large number of requests with limited GPU resources is becoming a key factor in determining actual implementation costs and operational efficiency.” He added, “This agreement is significant in that it combines Nota Inc.’s AI model lightweighting and inference/serving optimization technologies with actual enterprise Agent AI applications.”
He continued, “We will support the efficient use of AI agents even in environments with high security and infrastructure operational constraints, such as the financial sector, and we will expand our business supplying optimization software and AI execution environments through collaboration with various industry partners.”
Ahead of the 2027 College Scholastic Ability Test (CSAT) scheduled for the 19th, SamsungElectronics and LGELECTRONICS are inspecting the central air conditioning systems installed in test centers and …
CU is targeting the fanbase of the idol group RESCENE by selling character merchandise following the release of RESCENE bread. With the bread—developed in collaboration with the members—having sold ov…
#Nota Inc., a company specializing in optimizing AI inference, is setting its sights on the artificial intelligence (AI) coding agent market, targeting use even within the financial sector’s closed ne…