AutoPrecisePrompts: Automated LLM-based Prompt Engineering for Structured Data Processing

Abstract

Processing and manipulating structured data with Language Models has become vital for various use cases. However, not all language models may follow the expected output consistently, necessitating reattempts until the requirements are met. Repeated queries increase the resources and time required to process the information. Such problems necessitate effective Prompt Engineering and testing across various test cases. Prompt Engineering, when performed without automation, requires a larger workforce and significant time and resources. An alternative approach, such as Prompt Tuning, introduces further challenges. To solve all the challenges, this research proposes an AI-driven automated prompt optimization system designed to enhance the accuracy of prompts for various AI applications, using minimal time and resources. By iteratively testing prompts using a smaller language model and adjusting the prompt with the help of a Large Language Model until optimal performance is achieved, the system automates the process of optimizing prompts. Without requiring a training process before optimization, this approach ensures the reusability and transparency of optimized prompts to use across different language models. The system uses the expected output to offer a way for organizations to overcome the difficulties associated with manual-only prompt engineering. The system offers a solution to create concise, high-quality prompts that yield the desired accuracy. During the experiment, the system achieved the expected accuracy using only two iterations. The prompt led to satisfactory accuracy using multiple Language Models, proving the reusability of the prompt