Prompt Engineering for Generative AI Part 2

A prompt engineer could be anyone who has ever queried a generative AI model, and soon enough, that might be everyone! Natural language processing models mean that anyone can engineer a prompt without specialized knowledge. But what if your vision is more specific? What if you want more control over the output? This article will discuss the methods that prompt engineers use to craft the most effective input.

• In Part 1, we gave an overview of generative AI and discussed the relationship between generative AI and prompt engineering.
• In Part 2, we talk about the role of prompt engineering, who prompt engineers are, and the skills and knowledge behind a successful prompt.

Who are prompt engineers?

Communication between humans and natural language processing generative AI happens at the prompt, and getting the prompt right is important. The rise of platforms for generating, exchanging, inspiring, and even selling prompts for generative AI illustrates the significance of prompt engineering and the growing demand for skilled prompt engineers^[1].

Skills and knowledge for prompt engineering

According to one study, “[generative AI] task performance depends significantly on the quality of the prompt used to steer the model, and most effective prompts have been handcrafted by humans.^[2]” So, how does a skilled prompt engineer steer the model?

Here are some common skills and knowledge likely to serve prompt engineers.

Understanding of the desired output

The most basic skill in prompt engineering is to know what you are asking for. While it seems obvious, a strong understanding of the task and the output is critical to a well-crafted prompt. When a prompt engineer understands the specific details of the product they are asking for or what they want to accomplish, they can incorporate those details into the prompt.

Mastery of language

A good prompt uses clear language that the model can understand. A prompt engineer should also know how to provide the context and detail to the level required for the output. Depending on the use case, mastery of language might include the ability to describe the voice of a written piece or the mood conveyed by an image. A prompt engineer should understand how nuances in language change the meaning of the prompt.

Analytical skills

Prompt engineers test and adjust their prompts, and analyze the effect of those adjustments on the model’s output. Prompt engineers should be able to identify patterns in the output and how those patterns relate to the model’s knowledgebase.

Programming skills

The ability to understand the machine learning algorithms behind the generative AI model can help the prompt engineer understand how the model uses its dataset and processes the prompt. When co-programming with generative AI, the ability to read the code also supports the debugging and analysis stage. Programming skills might also be valuable to prompt engineers because programming is a way to apply logic and precision to language, which is an important part of composing prompts. 

Familiarity with natural language processing

Prompt engineers who understand how generative AI models process and generate natural language can more effectively steer the output of the model. Similar to programming skills, knowing how the model “thinks” can help the prompt engineer identify problems with the output and how to adjust the prompt^[3,4,5].  

How does prompt engineering compare to programming?

With prompt engineering, language is used in an intentional way to optimize results, and small adjustments can change the output. Prompt engineering adds parameters and safeguards, specifies what to put in, what to leave out, and where to look. When viewed as a science, prompt engineering starts to look a lot like another programming language.

However, even without precision engineering, generative AI like ChatGPT can converse in natural language, in multiple languages, and as some have pointed out, it seems to intuit meaning much like a human can^[6]. Everyday users can start creating with generative AI without specific engineering knowledge.

Generative AI can also write code from natural language prompts. Much like the artists who co-create with generative AI that were discussed in Part 1, programmers can co-program by describing the function of a program to generative AI. For example, here is a program generated by ChatGPT to adjust recipes for elevation.

Screenshot of ChatGPT by Lydia Husser.

While this program is simplistic, it raises some intriguing possibilities. In the same way that an artist might modify a prompt to perfect an AI-generated image, a software engineer with robust programming knowledge might use generative AI to code and debug more sophisticated applications.

Will prompt engineering replace traditional programming?

It does seem possible that prompt engineering could overtake traditional software programming as more companies incorporate AI into business tools. However, that possibility does not automatically devalue programming knowledge. As discussed in the previous sections, programming knowledge is advantageous in prompt design because it gives the prompt engineer a deep understanding of how and why changing the prompt changes the output.

Another aspect of this is that prompt engineering may become a standard skill for employees, much like sending an email or using a search engine. Microsoft has predicted that generative AI will transform business, and plans to incorporate aspects of generative AI into their software tools^[7].

Lessons learned from prompt engineering

The art and science of prompt engineering has had an effect on generative AI, which has shown us the influence that our patterns of communication have on machine learning models. Prompt engineering has taught us some interesting things about generative AI and how we collaborate with it.

The importance of expertise

When a prompt engineer has more knowledge about what the output should be, the limitations of the generative AI model, how it handles natural language, and the algorithms that process the dataset, the engineer can be more specific when crafting the prompt, resulting in output that is consistent and more aligned with the original vision.

The importance of nuance in language

A sophisticated natural language processing generative AI model can discern subtlety in language, use context to derive meaning, and incorporate abstractions like feeling and voice. A prompt engineer who understands the impact of language has greater control over the model’s output.

The effect of bias

The output from generative AI is influenced by explicit and implicit information in the prompt. Prompts that enforce biases have been shown to produce biased output from generative AI^[8]. Prompt engineers should be aware of their own biases and how they might sway the output.

The power of co-creation

Generative AI has the potential to extend human creativity by increasing our efficiency and removing barriers from curiosity and experimentation. Generative AI is useful only with human input. When humans direct the output of generative AI, the creative possibilities seem limitless.

Conclusion

Prompt engineering for generative AI is the simple act of telling the AI what you want, which might be the code for a program, an analysis of data, a prediction, a text response, or a work of art. But, as we have shown in this article, telling the AI what you want is just the beginning. The most successful prompt engineers have the tools and knowledge to craft prompts that achieve well-defined output. Prompt engineering is also a process of continuous refinement, and the more we interact with generative AI, the more we discover the ways in which the output is influenced by the input. 

If you haven’t read Part 1 yet, we recommend checking it out for an overview of generative AI and some specific use cases for prompt engineering. 

About the sponsor: Mouser

Mouser Electronics is a worldwide leading authorized distributor of semiconductors and electronic components for over 1,200 manufacturer brands. They specialize in the rapid introduction of new products and technologies for design engineers and buyers. Their extensive product offering includes semiconductors, interconnects, passives, and electromechanical components.

References

[1] Peter, A. C. (2023). Axios: AI's rise generates new job title: Prompt engineer. Newstex. Accessed 14 March 2023 from https://www.proquest.com/blogs-podcasts-websites/axios-ais-rise-generates-new-job-title-prompt/docview/2778629964/se-2

[2] Zhou, Y. et al (2022). Large Language Models are Human-Level Prompt Engineers. Arxiv. Accessed 14 March 2023 from https://arxiv.org/abs/2211.01910.

[3] Gao et al. (2021). Fine-tuning Language Models from Human Preferences. Accessed 15 March 2023 from https://arxiv.org/abs/1909.08593.

[4] Dathathri, S. et al. Plug and Play Language Models: A Simple Approach to Controlled Text Generation. Accessed 15 March 2023 from https://arxiv.org/abs/1912.02164 

[5] Hardesty, Larry. AAAI: Prompt engineering and reasoning in the spotlight. Accessed 15 March 2023 from https://www.amazon.science/blog/aaai-prompt-engineering-and-reasoning-in-the-spotlight 

[6] Future, B. T. (2021). The Problem with Prompt Engineering. Accessed 14 March 2023 from https://bakztfuture.substack.com/p/the-problem-with-prompt-engineering.

[7] Schechner, S. (2023, Jan 17). Microsoft plans to build OpenAI, ChatGPT features into all products. Wall Street Journal (Online) Retrieved from https://www.proquest.com/newspapers/microsoft-plans-build-openai-chatgpt-features/docview/2765930434/se-2

[8] University of Waterloo. ChatGPT and Generative AI: Potential for bias based on prompt. Accessed 15 March 2023 from https://subjectguides.uwaterloo.ca/chatgpt_generative_ai/chatgptbias.